Metraipolis › archive › dc_ce44223cede0

Adaptive Finite-Budget Training for CVaR Risk-Aware Q-Learning — arXiv:2608.04305

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.04305
License (as archived)
CC0-1.0
Archived
2026-10-31T11:34:45.056Z
SHA-256 of the archived text
baa554755229160b990beee549838fec0598a2364bff1c03f7b1b4e8380b36a7
Stable URL of this page
https://metraipolis.com/documento/dc_ce44223cede0

How to cite

Adaptive Finite-Budget Training for CVaR Risk-Aware Q-Learning — arXiv:2608.04305 — arXiv (en), https://arxiv.org/abs/2608.04305 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ce44223cede0 (Metraipolis archive dc_ce44223cede0, sha256 baa554755229160b990beee549838fec0598a2364bff1c03f7b1b4e8380b36a7)
BibTeX
@misc{dcce44223cede0,
  title = {Adaptive Finite-Budget Training for CVaR Risk-Aware Q-Learning — arXiv:2608.04305},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.04305},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ce44223cede0; sha256 baa554755229160b990beee549838fec0598a2364bff1c03f7b1b4e8380b36a7},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.04305. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Risk-aware Q-learning (RaQL) provides a model-free, two-timescale estimator for dynamic risk objectives, but its finite-budget behavior remains fragile: fixed inner-loop hyperparameters can produce unstable value estimates, persistent Bellman residuals, and inefficient sample reuse. This paper proposes an adaptive training controller for Conditional Value-at-Risk (CVaR) RaQL and evaluates it on a daily Bitcoin trading task. The controller preserves the original CVaR estimator and Bellman fixed point; instead, it redesigns the training procedure through six coordinated mechanisms: per-cell inner-step sizing, outer-rate-matched decay synchronization, a short early correction for the VaR-like inner variable, a coverage-first-then-greedy sample allocation rule, progressive suffix aggregation of mature inner estimates, and data-driven calibration of key scales from online-observable quantities. Across 20 random seeds and 856,000 inner-transition samples, the controller reduces the mean empirical CVaR Bellman residual by approximately 85% relative to the fixed-parameter baseline (MeanBEQ: 1.2202 to 0.1854; MeanBEV: 1.1624 to 0.0535) and maintains stability across CVaR levels, discount factors, and training budgets. On the chronological out-of-sample test set, the learned policy attains a Sharpe ratio of 0.9281 with a maximum drawdown of 6.46% after transaction costs. Although buy-and-hold yields a higher cumulative return (35.43% vs. 23.61%), the adaptive policy achieves far lower volatility (9.57% vs. 47.93%), drawdown, and CVaR loss. These results demonstrate that adaptive finite-budget training design, applied solely to the training procedure without altering the risk objective, can materially improve the reliability and risk-adjusted performance of risk-aware Q-learning in financial applications.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-06 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Risk Management (q-fin.RM).
Pagina del lavoro: https://arxiv.org/abs/2608.04305 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T11:24:48.771Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-06?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).