Metraipolis › archive › dc_29f8b70396bd

Reach or Solve? Deep Diving into Agentic RL Gains with Checkpoint Handoffs — arXiv:2609.19636

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.19636
License (as archived)
CC0-1.0
Archived
2026-10-10T03:20:21.508Z
SHA-256 of the archived text
75fe78ec02fce462ab975f5e4b11c6eb52643194a24a781b9fc8d9d385bef0a4
Stable URL of this page
https://metraipolis.com/documento/dc_29f8b70396bd

How to cite

Reach or Solve? Deep Diving into Agentic RL Gains with Checkpoint Handoffs — arXiv:2609.19636 — arXiv (en), https://arxiv.org/abs/2609.19636 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_29f8b70396bd (Metraipolis archive dc_29f8b70396bd, sha256 75fe78ec02fce462ab975f5e4b11c6eb52643194a24a781b9fc8d9d385bef0a4)
BibTeX
@misc{dc29f8b70396bd,
  title = {Reach or Solve? Deep Diving into Agentic RL Gains with Checkpoint Handoffs — arXiv:2609.19636},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.19636},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_29f8b70396bd; sha256 75fe78ec02fce462ab975f5e4b11c6eb52643194a24a781b9fc8d9d385bef0a4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.19636. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Reinforcement learning (RL) is widely used to improve language-model agents, and its gains are usually measured by final task success. However, an agent's earlier actions shape the states in which its later decisions are made, so final task success conflates the ability to reach useful states with the ability to complete the task once there. Comparing agents only on the states each one reaches does not separate the two, since each agent is then scored on states selected by its own actions. To address this conflation, we introduce checkpoint handoff, an evaluation protocol that decouples reaching from completing without retraining. One checkpoint acts as a reacher up to a handoff point, and another continues as the solver from the same replayed history. In detail, (1) Reach measures how often a reacher arrives at states that a replayable environment verifies to be a fixed number of actions from success, and (2) Solve measures how often a solver completes the task from identical cloned copies of those states. Crossing supervised fine-tuning (SFT) and RL checkpoints in both roles across two benchmarks and two independently released training pipelines, we find that the gain from switching the solver from SFT to RL is consistently larger when RL is the reacher, at all three model scales on TravelPlanner and on both ALFWorld splits. Further analyses on ALFWorld show that RL improves both Reach and Solve. The solver gain is larger under an RL reacher because RL reaches solvable states more often, and separately measured Reach and Solve gaps recover most of this difference. Because handoff only requires replaying one checkpoint's history under another, agentic RL evaluations can report arrival and completion alongside final success.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-18 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.19636 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-10T03:04:13.622Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).