Metraipolis › archive › dc_12e4a7ffb74a

My FAULT: Self-Diagnosis as Credit Assignment in Self-Evolving Agentic Reinforcement Learning — arXiv:2610.01161

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.01161
License (as archived)
CC0-1.0
Archived
2026-10-06T13:55:02.848Z
SHA-256 of the archived text
861470df39829a3a7c4317251ca63d06579b3d2178a0c3f6f3fbe8d8f9b104de
Stable URL of this page
https://metraipolis.com/documento/dc_12e4a7ffb74a

How to cite

My FAULT: Self-Diagnosis as Credit Assignment in Self-Evolving Agentic Reinforcement Learning — arXiv:2610.01161 — arXiv (en), https://arxiv.org/abs/2610.01161 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_12e4a7ffb74a (Metraipolis archive dc_12e4a7ffb74a, sha256 861470df39829a3a7c4317251ca63d06579b3d2178a0c3f6f3fbe8d8f9b104de)
BibTeX
@misc{dc12e4a7ffb74a,
  title = {My FAULT: Self-Diagnosis as Credit Assignment in Self-Evolving Agentic Reinforcement Learning — arXiv:2610.01161},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.01161},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_12e4a7ffb74a; sha256 861470df39829a3a7c4317251ca63d06579b3d2178a0c3f6f3fbe8d8f9b104de},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.01161. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Agentic reinforcement learning (RL) has emerged as a powerful approach for training large language model agents on multi-step tasks, yet reliance on terminal outcome rewards creates two credit-assignment problems, particularly in long-horizon tasks. First, same-outcome rollout groups provide no learning signal from terminal rewards. Second, terminal rewards provide only trajectory-wide feedback, making it difficult to identify which decisions caused a failure. Recent work supplements terminal rewards with finer-grained information from trajectory analysis, such as natural-language reflections on intermediate decisions and errors. However, natural-language diagnoses are difficult to use directly for credit assignment: their error claims may be unreliable, and they do not quantify how much each error should affect learning. We propose Self-Diagnosis-guided Terminal Credit Redistribution (FAULT), which turns diagnosed errors into explicit step-level credit anchored by terminal outcomes. FAULT checks diagnostic evidence and learns relative error costs from task outcomes. During training, the policy and self-diagnoser co-evolve, while error costs are updated online from recent outcomes. On ALFWorld, FAULT recovers learning signals from same-outcome groups, reaching 95% signal coverage versus 41% for GRPO and 72% for GiGPO, while better localizing credit to specific error steps. Across two model scales, FAULT delivers strong. improvements on the long-horizon ALFWorld and WebShop tasks while remaining competitive on short-horizon Search-based QA.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-02 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.01161 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T12:15:34.686Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).