Metraipolis › archive › dc_192fdea387f0

Training Advisors for LLM Agents from Task Outcomes — arXiv:2610.09858

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.09858
License (as archived)
CC0-1.0
Archived
2026-10-08T11:11:32.934Z
SHA-256 of the archived text
1d4b7d8d33e5b12f9e9adca91e36dad6d4cc5a37785338ddd5bb7adf51033a2b
Stable URL of this page
https://metraipolis.com/documento/dc_192fdea387f0

How to cite

Training Advisors for LLM Agents from Task Outcomes — arXiv:2610.09858 — arXiv (en), https://arxiv.org/abs/2610.09858 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_192fdea387f0 (Metraipolis archive dc_192fdea387f0, sha256 1d4b7d8d33e5b12f9e9adca91e36dad6d4cc5a37785338ddd5bb7adf51033a2b)
BibTeX
@misc{dc192fdea387f0,
  title = {Training Advisors for LLM Agents from Task Outcomes — arXiv:2610.09858},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.09858},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_192fdea387f0; sha256 1d4b7d8d33e5b12f9e9adca91e36dad6d4cc5a37785338ddd5bb7adf51033a2b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.09858. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language model agents tackle multi-step tasks by interleaving reasoning and tool calls with observations from the environment. Prior work has shown that natural-language feedback can help these agents revise their decisions during task execution. We introduce Caddie, a method for training critics to provide natural-language analysis and advice as agents work through a task. Unlike approaches that rely on step-level labels or reference critiques, Caddie learns from whether the agent ultimately succeeds after receiving the critic's feedback. We optimize the critic through reinforcement learning while keeping the base model frozen. Trained on multi-hop question answering with a single base model, our Qwen3-4B critic improves success rates across four base models of different scales and architectures, including three not used during critic training. On the MuSiQue benchmark, the trained critic improves Qwen3-4B's success rate by more than 25 percentage points, surpassing the performance of Kimi K3 without a critic. The same critic also yields gains on out-of-domain interactive benchmarks, including $\tau^3$ and DeepDive, with no additional training. Our results show that agents can decide when to seek help from a critic at inference time and that outcome-based critic training can produce guidance that transfers across base models and task domains.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.09858 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T10:14:06.133Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).