Metraipolis › archive › dc_248ef31fa3e2

Beyond Corrected Actions: Training LLM Agents on Textual Critiques — arXiv:2610.19367

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.19367
License (as archived): CC0-1.0
Archived: 2026-10-15T20:02:36.748Z
SHA-256 of the archived text: f225698dea1c2754c19d749e6cff03e64ec26145ebe216bfc0bdf1f929514f9b
Stable URL of this page: https://metraipolis.com/documento/dc_248ef31fa3e2

How to cite

Beyond Corrected Actions: Training LLM Agents on Textual Critiques — arXiv:2610.19367 — arXiv (en), https://arxiv.org/abs/2610.19367 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_248ef31fa3e2 (Metraipolis archive dc_248ef31fa3e2, sha256 f225698dea1c2754c19d749e6cff03e64ec26145ebe216bfc0bdf1f929514f9b)

BibTeX

@misc{dc248ef31fa3e2,
  title = {Beyond Corrected Actions: Training LLM Agents on Textual Critiques — arXiv:2610.19367},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.19367},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_248ef31fa3e2; sha256 f225698dea1c2754c19d749e6cff03e64ec26145ebe216bfc0bdf1f929514f9b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.19367. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) are increasingly used for multi-turn agentic tasks that require interactions with users and environments over extended sequences of actions. A common approach to improving LLM agents is to use more capable expert models to provide training supervision, either by behavior cloning the expert's trajectories or by using the expert to correct the actions in trajectories generated by the weaker student model. However, expert LLMs can provide richer supervision than actions alone: they can directly generate textual critique that explain why specific actions are incorrect. We study whether training on such feedback benefits LLM agents and propose Critique-Augmented Agentic Fine-Tuning (CAFT), a framework that elicits textual expert feedback on student-generated trajectories. For each incorrect action, the expert yields both a corrected action and a textual critique explaining the student's mistake--which we then use as the training target for fine-tuning. Across four student models from the Nemotron 3 and Gemma 4 families on the agentic domains $\tau^2$-Bench Airline+Retail and WebShop, adding critiques to action-level supervision improves performance by 8% on average over action-only distillation. Additionally, CAFT checkpoints can also provide a better warm start for reinforcement learning than expert-trajectory distillation. Our results demonstrate that textual critiques can strengthen supervision for multi-turn LLM agents.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.19367 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T19:35:48.332Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).