Metraipolis › archive › dc_ee4682db3c81
BCPPO: Bachelier-Inspired Constrained Proximal Policy Optimization for Tail-Risk-Aware Safe Reinforcement Learning — arXiv:2608.30283
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.30283
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T07:52:33.088Z
- SHA-256 of the archived text
eb18a0a2b3d48b3e4791197944445b128d6612fed7e9cf00ea695193450e4034- Stable URL of this page
https://metraipolis.com/documento/dc_ee4682db3c81
How to cite
BCPPO: Bachelier-Inspired Constrained Proximal Policy Optimization for Tail-Risk-Aware Safe Reinforcement Learning — arXiv:2608.30283 — arXiv (en), https://arxiv.org/abs/2608.30283 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ee4682db3c81 (Metraipolis archive dc_ee4682db3c81, sha256 eb18a0a2b3d48b3e4791197944445b128d6612fed7e9cf00ea695193450e4034)
BibTeX
@misc{dcee4682db3c81,
title = {BCPPO: Bachelier-Inspired Constrained Proximal Policy Optimization for Tail-Risk-Aware Safe Reinforcement Learning — arXiv:2608.30283},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.30283},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ee4682db3c81; sha256 eb18a0a2b3d48b3e4791197944445b128d6612fed7e9cf00ea695193450e4034},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.30283. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Expected-cost constraints can still permit rare, high-cost events. Monte Carlo conditional value at risk (CVaR) gradients can be noisy at high confidence, whereas critics that model an outcome distribution add complexity. We propose BCPPO (Bachelier-Inspired Constrained Proximal Policy Optimization), a proximal policy optimization (PPO) method. Separately initialized cost-prediction networks (critics), trained with random sample masks, produce disagreement that marks predictions sensitive to which state-action regions occur in the training data and to critic training. A Bachelier formula for the expected amount above a reference level converts this disagreement into a smooth policy-update penalty. Gradients from this penalty do not alter the critics, so temporal-difference (TD) critic learning is unchanged. A saturation-aware controller adjusts the mean-cost penalty and stops accumulated error from growing while that penalty is clipped. Deployment retains only the policy network. The disagreement penalty is neither a tail-event probability nor a guaranteed error bound, and it provides no safety guarantee. Across 175 runs with shared tasks, costs, budgets, training steps, and evaluation seeds, no comparator attains both higher mean return and lower mean CVaR than BCPPO in any task. On Push1, BCPPO has no lower return and no higher CVaR than every comparator, with at least one strict gain. These results support a practical balance among reward, caution around cost predictions that vary across trained critics, and policy-only deployment. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.AI). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2608.30283 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T07:28:26.848Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).