Metraipolis › archive › dc_4939cccb8ae9

Recursive Value Learning for Long-Horizon Offline Goal-Conditioned RL — arXiv:2609.02237

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.02237
License (as archived)
CC0-1.0
Archived
2026-10-17T04:59:27.354Z
SHA-256 of the archived text
df8deb466e4a9cf2105ee7ace4bfd97ef4b527006eed35afb06b893a882747e9
Stable URL of this page
https://metraipolis.com/documento/dc_4939cccb8ae9

How to cite

Recursive Value Learning for Long-Horizon Offline Goal-Conditioned RL — arXiv:2609.02237 — arXiv (en), https://arxiv.org/abs/2609.02237 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4939cccb8ae9 (Metraipolis archive dc_4939cccb8ae9, sha256 df8deb466e4a9cf2105ee7ace4bfd97ef4b527006eed35afb06b893a882747e9)
BibTeX
@misc{dc4939cccb8ae9,
  title = {Recursive Value Learning for Long-Horizon Offline Goal-Conditioned RL — arXiv:2609.02237},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.02237},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4939cccb8ae9; sha256 df8deb466e4a9cf2105ee7ace4bfd97ef4b527006eed35afb06b893a882747e9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.02237. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Scaling offline goal-conditioned reinforcement learning (GCRL) to long-horizon tasks is difficult because (1) long-range value learning depends on shorter-range estimates that may still be inaccurate, and (2) max-based value backups can amplify overestimation through repeated propagation. We propose DCRL (Divide-and-Conquer RL), which recursively decomposes each trajectory segment into a balanced binary tree and trains the values from leaves to root. Each parent is therefore updated only after its children, using an exact factorization of the observed route rather than selecting among noisy alternatives. Since this objective learns values along demonstrated routes that are not necessarily optimal, DCRL jointly propagates values across trajectories to discover shorter routes. Thanks to the balanced binary tree, DCRL reduces worst-case bootstrap depth from linear to logarithmic, and this shorter dependency structure empirically corresponds to much slower error accumulation. Across diverse goal-reaching tasks, DCRL substantially outperforms prior flat offline GCRL methods, and on the five most challenging long-horizon OGBench tasks, it improves the best prior average score from 55 to 64, surpassing all flat and hierarchical baselines.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-03 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2609.02237 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T04:09:43.318Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-03?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).