Metraipolis › archive › dc_e096744ff1ce
TrajEvolve: Trajectory Evolution for Reinforcement Learning with Hindsight Credit Assignment — arXiv:2610.28060
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.28060
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-21T07:06:59.381Z
- SHA-256 of the archived text
e7983611df4cd407c9b6c1cc1c0adec9b3c0dcfed078f82301d8e93c36f3f85c- Stable URL of this page
https://metraipolis.com/documento/dc_e096744ff1ce
How to cite
TrajEvolve: Trajectory Evolution for Reinforcement Learning with Hindsight Credit Assignment — arXiv:2610.28060 — arXiv (en), https://arxiv.org/abs/2610.28060 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e096744ff1ce (Metraipolis archive dc_e096744ff1ce, sha256 e7983611df4cd407c9b6c1cc1c0adec9b3c0dcfed078f82301d8e93c36f3f85c)
BibTeX
@misc{dce096744ff1ce,
title = {TrajEvolve: Trajectory Evolution for Reinforcement Learning with Hindsight Credit Assignment — arXiv:2610.28060},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.28060},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e096744ff1ce; sha256 e7983611df4cd407c9b6c1cc1c0adec9b3c0dcfed078f82301d8e93c36f3f85c},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.28060. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
The Reinforcement Learning (RL) paradigm has achieved remarkable success in enhancing the reasoning capabilities of Large Language Models (LLMs). However, when applied to long-horizon tasks, it faces severe challenges: i) Due to the high task complexity, the model struggles to sample enough successful trajectories, resulting in slow convergence. ii) Relying solely on sparse final rewards causes all steps to be updated indiscriminately, encouraging redundant step generation. When both the quantity and quality of successful trajectories become difficult to guarantee, the performance is severely constrained. This dilemma stems from the limitations: existing RL paradigms rely entirely on the reasoning capability of base models, lacking the ability to actively optimize trajectories. To address the above issues, we propose a novel trajectory evolution paradigm with hindsight credit assignment, termed TrajEvolve. TrajEvolve not only relies on the trajectory inferred by the model but also generates ``evolutionary trajectories'' by refining low-quality steps in historical trajectories, thereby exposing the policy to high-quality compositions that lie within the base model's support but are rarely sampled within practical training budgets. Furthermore, TrajEvolve reformulates step-level continuous credit assignment as a binary classification task, whether each step is necessary for task completion. This simplification not only reduces the learning difficulty but also enables step labels to be obtained through counterfactual verification, achieving precise identification of redundant steps. Extensive experiments on two datasets demonstrate the superiority of the proposed method. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.28060 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-21T07:05:43.463Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).