Metraipolis › archive › dc_423a7b453eaa
Risk-Sensitive Reinforcement Learning with Smoothed Quantile Objectives — arXiv:2608.22227
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.22227
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T08:25:46.931Z
- SHA-256 of the archived text
c5791247bb6aa4d5046bdcde8f892db89bd10e08ac1f4831d739f359dd3767a4- Stable URL of this page
https://metraipolis.com/documento/dc_423a7b453eaa
How to cite
Risk-Sensitive Reinforcement Learning with Smoothed Quantile Objectives — arXiv:2608.22227 — arXiv (en), https://arxiv.org/abs/2608.22227 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_423a7b453eaa (Metraipolis archive dc_423a7b453eaa, sha256 c5791247bb6aa4d5046bdcde8f892db89bd10e08ac1f4831d739f359dd3767a4)
BibTeX
@misc{dc423a7b453eaa,
title = {Risk-Sensitive Reinforcement Learning with Smoothed Quantile Objectives — arXiv:2608.22227},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.22227},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_423a7b453eaa; sha256 c5791247bb6aa4d5046bdcde8f892db89bd10e08ac1f4831d739f359dd3767a4},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.22227. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Reinforcement Learning (RL) has achieved tremendous success in recent years. However, the classical foundations of RL do not account for the risk sensitivity of the objective function, which is critical in various fields, including healthcare, finance, etc. A popular approach to incorporate risk sensitivity is to optimize a specific quantile of the cumulative reward distribution. However, exact quantile objectives are non-smooth and can change abruptly under small perturbations of the return distribution, making them difficult to optimize reliably when the transition model must be learned from data. Motivated by this instability, we develop UCB-BQRL, a model-based optimistic learning algorithm that maintains confidence sets for the transition kernel and plans using a lower-buffered quantile criterion. The buffered criterion smooths the exact quantile objective by averaging nearby lower quantiles, thereby improving stability under transition-estimation error. To compute the buffered-quantile policy at each episode, we introduce EVI-BQ, an exact dynamic-programming procedure. We establish a high-probability regret bound for UCB-BQRL, which up to logarithmic factors scales as $\mathcal{O}(\mathrm{e}^{\tau/\rho_\tau}+H^2\sqrt{SAT})$, where $\rho_\tau$ is denoted as the root-level left-plateau threshold, which is a problem-dependent constant. Further, we establish an information-theoretic lower bound of $\Omega(H/\rho_\tau\sqrt{AT})$ for the regret of any algorithm dealing with a quantile objective function. Finally, we prove that the exact point-quantile evaluation and exact lower-buffered quantile evaluation are PP-hard under polynomial-time Turing reductions, even for a fixed policy in a two-state, one-action finite-horizon MDP.
Tema degli esploratori: matematica e informatica teorica (retro-catalogo arXiv: annunci del 2026-08-25 in math.OC).
Categorie arXiv: Machine Learning (cs.LG) ; Optimization and Control (math.OC).
Pagina del lavoro: https://arxiv.org/abs/2608.22227 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T08:18:05.717Z dalla pagina https://arxiv.org/catchup/math.OC/2026-08-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).