Metraipolis › archive › dc_2a1ae83a7367
Grounded Credit Assignment in Reinforcement Learning for Multi-Turn Search Agents — arXiv:2610.23148
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.23148
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T08:48:37.725Z
- SHA-256 of the archived text
a59c96bf01a3257948c0ffc6b256d342aa85a2df58b345f21ffbea5d12c5403c- Stable URL of this page
https://metraipolis.com/documento/dc_2a1ae83a7367
How to cite
Grounded Credit Assignment in Reinforcement Learning for Multi-Turn Search Agents — arXiv:2610.23148 — arXiv (en), https://arxiv.org/abs/2610.23148 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2a1ae83a7367 (Metraipolis archive dc_2a1ae83a7367, sha256 a59c96bf01a3257948c0ffc6b256d342aa85a2df58b345f21ffbea5d12c5403c)
BibTeX
@misc{dc2a1ae83a7367,
title = {Grounded Credit Assignment in Reinforcement Learning for Multi-Turn Search Agents — arXiv:2610.23148},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.23148},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2a1ae83a7367; sha256 a59c96bf01a3257948c0ffc6b256d342aa85a2df58b345f21ffbea5d12c5403c},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.23148. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Outcome-reward reinforcement learning such as GRPO gives every token of a search-agent rollout the same advantage, although the rollout exercises two capabilities: finding evidence and using it. We show that accuracy improves without grounding: Search-R1 raises accuracy on all seven QA benchmarks, yet on Bamboogle the evidence-supported share of correct answers falls from 84.0% to 42.9%. We attribute this to GRPO's credit assignment: rollouts that retrieve answer-bearing evidence are favoured in aggregate only in proportion to their accuracy gain over the rest, a grounding gap that answering from memory or misusing evidence shrinks or reverses. To address this, we propose Grounded Credit Assignment (GCA), which uses verifiable sub-outcomes (evidence retrieval and answer correctness) to correct search and answer credit separately: a wrong answer is blamed more than the search that found the evidence, and a correct answer that bypasses retrievable evidence is credited less. Never negative on evidence-finding search, the correction does not shrink with the grounding gap; capped at 10% of each token's advantage, it never overrides the outcome signal. Without extra models, rollouts, or training stages, GCA improves average EM over Search-R1 by 5.7 (Qwen2.5-3B) and 6.3 points (Qwen3-30B-A3B) and raises this share from 76.9% to 93.4%. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.23148 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T08:42:40.261Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).