Metraipolis › archive › dc_13cf82490e4e

Granularity-Adaptive Credit Assignment for Long-Horizon LLM Agent Reinforcement Learning — arXiv:2609.12424

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.12424
License (as archived)
CC0-1.0
Archived
2026-10-16T03:15:41.869Z
SHA-256 of the archived text
0173ec97ab99bda5edf458529a3dcf23e2e3582af3d3479c0e93ac6d06f8a7f2
Stable URL of this page
https://metraipolis.com/documento/dc_13cf82490e4e

How to cite

Granularity-Adaptive Credit Assignment for Long-Horizon LLM Agent Reinforcement Learning — arXiv:2609.12424 — arXiv (en), https://arxiv.org/abs/2609.12424 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_13cf82490e4e (Metraipolis archive dc_13cf82490e4e, sha256 0173ec97ab99bda5edf458529a3dcf23e2e3582af3d3479c0e93ac6d06f8a7f2)
BibTeX
@misc{dc13cf82490e4e,
  title = {Granularity-Adaptive Credit Assignment for Long-Horizon LLM Agent Reinforcement Learning — arXiv:2609.12424},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.12424},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_13cf82490e4e; sha256 0173ec97ab99bda5edf458529a3dcf23e2e3582af3d3479c0e93ac6d06f8a7f2},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.12424. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Long-horizon language-model agents trained with reinforcement learning oftenreceive sparse outcome rewards that do not reveal which decisions along a tra-jectory deserve credit. Episode-level advantages provide coarse trajectory-widecredit, while step-level comparisons offer finer resolution with context-dependentestimation noise. We propose Granularity-Adaptive Credit Assignment (GACA),a critic-free method that adaptively mixes episode- and step-level credit for eachdecision during policy optimization. GACA normalizes the sampled response'smean per-token negative log-likelihood (NLL) within each trajectory and uses theresulting criticality score to determine the step-specific mixture. The computationreuses rollout log-probabilities without additional training rollouts or model eval-uations. Our analysis characterizes optimal score-dependent mixing and derivesconditions linking expected NLL to a lower bound on the preferred step-levelweight. Across ALFWorld and WebShop with 1.5B and 7B backbones, GACAachieves the highest reported mean success rates among the compared methods,while introducing negligible additional computation.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-14 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.12424 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T03:05:44.043Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).