Metraipolis › archive › dc_5543d15dcf41

Rare Event Constrained Reinforcement Learning with Large Deviations — arXiv:2611.02055

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.02055
License (as archived)
CC0-1.0
Archived
2026-11-03T18:40:37.935Z
SHA-256 of the archived text
146c6c6d43ffbe41ab8b3313366e795fbc66b6c621306dc7e4c9bc871bc8bd08
Stable URL of this page
https://metraipolis.com/documento/dc_5543d15dcf41

How to cite

Rare Event Constrained Reinforcement Learning with Large Deviations — arXiv:2611.02055 — arXiv (en), https://arxiv.org/abs/2611.02055 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5543d15dcf41 (Metraipolis archive dc_5543d15dcf41, sha256 146c6c6d43ffbe41ab8b3313366e795fbc66b6c621306dc7e4c9bc871bc8bd08)
BibTeX
@misc{dc5543d15dcf41,
  title = {Rare Event Constrained Reinforcement Learning with Large Deviations — arXiv:2611.02055},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.02055},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5543d15dcf41; sha256 146c6c6d43ffbe41ab8b3313366e795fbc66b6c621306dc7e4c9bc871bc8bd08},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02055. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We study finite-horizon reinforcement learning with a safety constraint on the probability of reaching a failure set. In contrast to the well-studied safe reinforcement learning literature, we consider problems where the probability of reaching the failure set is vanishingly small, but the consequences of reaching the failure set are catastrophic. Direct simulation and estimation of the optimal policy is naturally ineffective when the failure probability is extremely small. Moreover, on a real system the controller can only choose actions, and this is not enough to make failures frequent. In contrast to the rare event simulation literature, our goal is not to estimate the probability of the rare event occurrence, but to upper bound it such that the failure constraint is satisfied. Thus, instead of sampling failures, we use the large deviations principle. We define a damage variable that on exceeding a certain threshold indicates failure. We study the cumulant generating function (CGF) as its positive root provides an upper bound on the failure probability via Cramer's theorem. We extend this approach to Markov decision processes with hitting times through state-action CGFs, and give finite-horizon and state-dependent versions of the bound. The state-action CGFs do not depend on the policy and can be estimated from regular transitions, so the bound can be checked for any policy a priori. This gives a policy improvement algorithm in which every policy satisfies the failure constraint with high probability.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (stat.ML) ; Machine Learning (cs.LG); Systems and Control (eess.SY); Optimization and Control (math.OC); Probability (math.PR); Statistics Theory (math.ST).
Pagina del lavoro: https://arxiv.org/abs/2611.02055 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T18:12:54.752Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).