Metraipolis › archive › dc_19725224d84a

Deep Epistemic Value Functions for Optimistic Exploration — arXiv:2609.35525

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.35525
License (as archived)
CC0-1.0
Archived
2026-10-07T22:39:12.401Z
SHA-256 of the archived text
62efe6584f2711951848b5af10f4c57db6801cdc7199d6b6fc082c9b5bdce1b4
Stable URL of this page
https://metraipolis.com/documento/dc_19725224d84a

How to cite

Deep Epistemic Value Functions for Optimistic Exploration — arXiv:2609.35525 — arXiv (en), https://arxiv.org/abs/2609.35525 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_19725224d84a (Metraipolis archive dc_19725224d84a, sha256 62efe6584f2711951848b5af10f4c57db6801cdc7199d6b6fc082c9b5bdce1b4)
BibTeX
@misc{dc19725224d84a,
  title = {Deep Epistemic Value Functions for Optimistic Exploration — arXiv:2609.35525},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.35525},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_19725224d84a; sha256 62efe6584f2711951848b5af10f4c57db6801cdc7199d6b6fc082c9b5bdce1b4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.35525. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Principled exploration in reinforcement learning requires an agent to quantify its epistemic uncertainty and act to resolve it. Uncertainty over the value function provides a natural signal for exploration, yet existing deep approximations remain brittle and perform inconsistently. The central challenge is therefore to scale these ideas robustly. We conduct a systematic empirical study of how epistemic uncertainty is represented, propagated, and optimized in deep epistemic value functions, and uncover distinct failure modes along each of these axes. These findings motivate DEVOTE, a model-free reinforcement learning algorithm that controls how uncertainty generalizes beyond observed data, stabilizes its temporal propagation, and preserves adaptation to the resulting non-stationary exploration objective. Across reward-free exploration and challenging continuous-control tasks, DEVOTE reaches novel states more effectively and achieves higher task return than strong model-free and model-based exploration baselines. These results provide evidence that deep epistemic value functions are a promising path toward scalable, principled exploration.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.35525 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T22:14:28.467Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).