Metraipolis › archive › dc_4bf6ba2b3e85

Hindsight Credit Assignment over Trees for Agentic RL — arXiv:2611.02706

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.02706
License (as archived)
CC0-1.0
Archived
2026-11-03T18:20:04.349Z
SHA-256 of the archived text
d8dc8b1186139e5c8395b12742ad3cdd189d05534243ab48aff8ab10d11524b1
Stable URL of this page
https://metraipolis.com/documento/dc_4bf6ba2b3e85

How to cite

Hindsight Credit Assignment over Trees for Agentic RL — arXiv:2611.02706 — arXiv (en), https://arxiv.org/abs/2611.02706 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4bf6ba2b3e85 (Metraipolis archive dc_4bf6ba2b3e85, sha256 d8dc8b1186139e5c8395b12742ad3cdd189d05534243ab48aff8ab10d11524b1)
BibTeX
@misc{dc4bf6ba2b3e85,
  title = {Hindsight Credit Assignment over Trees for Agentic RL — arXiv:2611.02706},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.02706},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4bf6ba2b3e85; sha256 d8dc8b1186139e5c8395b12742ad3cdd189d05534243ab48aff8ab10d11524b1},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02706. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Reinforcement learning for long-horizon LLM agents faces a fundamental dilemma: successful experiences are essential for policy improvement, yet limited policy capabilities make such experiences difficult to discover. Sparse outcome rewards compound this challenge, leaving sampled trajectories with little relative feedback for learning. In this paper, we propose TriGRPO, a tree-based reinforcement learning framework that turns scarce successes into exploration and learning signals through hindsight credit assignment within and across rollout trees. TriGRPO constructs rollout trees through hindsight-guided branching: when an ongoing trajectory reaches a matched decision state, a successful action discovered during rollout initiates a sibling branch that continues autonomously, enabling further exploration and parent-sibling comparisons. Three complementary groups organize policy optimization over these trees: root groups capture within-prompt relative returns, node groups assign credit from parent-sibling outcome differences, and cross groups supplement sparse feedback with historical performance on related tasks. Experimental results demonstrate consistent gains in task performance and rollout efficiency across model scales, highlighting structured hindsight credit assignment as an effective approach to long-horizon agent learning.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.02706 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T18:09:53.704Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).