Metraipolis › archive › dc_4e60d59dbd79

VETTA: Coordinating Turn- and Token-Level Credit Assignment for Multi-Turn LLM Agents — arXiv:2610.08402

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.08402
License (as archived): CC0-1.0
Archived: 2026-10-07T11:41:46.826Z
SHA-256 of the archived text: 691129e5a24e132f33c49503eb415cfc2e9668f69c167d473760cb7f0af1b09a
Stable URL of this page: https://metraipolis.com/documento/dc_4e60d59dbd79

How to cite

VETTA: Coordinating Turn- and Token-Level Credit Assignment for Multi-Turn LLM Agents — arXiv:2610.08402 — arXiv (en), https://arxiv.org/abs/2610.08402 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4e60d59dbd79 (Metraipolis archive dc_4e60d59dbd79, sha256 691129e5a24e132f33c49503eb415cfc2e9668f69c167d473760cb7f0af1b09a)

BibTeX

@misc{dc4e60d59dbd79,
  title = {VETTA: Coordinating Turn- and Token-Level Credit Assignment for Multi-Turn LLM Agents — arXiv:2610.08402},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.08402},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4e60d59dbd79; sha256 691129e5a24e132f33c49503eb415cfc2e9668f69c167d473760cb7f0af1b09a},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.08402. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multi-turn LLM agents often receive sparse task feedback across several interactions, while generating each response token by token. This creates two related credit-assignment questions: which responses helped achieve the outcome, and which generation decisions mattered within each response? Existing methods typically focus on only one level: turn-level methods evaluate complete responses but do not distinguish the decisions within them; token-level methods can propagate feedback across turns but do not explicitly model credit for each response. These complementary limitations motivate learning credit at both levels and coordinating it in a single policy update. We introduce VETTA, a credit assignment method that jointly learns turn- and token-level values through separate heads on a shared lightweight critic. VETTA computes advantages along both temporal sequences and combines each turn advantage with a within-response-centered token residual for PPO updates. Furthermore, to reduce value-learning cost, the critic retains only early Transformer blocks from the pretrained checkpoint used to initialize the actor. On two challenging agent benchmarks, ALFWorld and WebShop, VETTA improves success rates over PPO by 37.5% and 22.3%, respectively, with Qwen2.5-1.5B-Instruct and achieves success rates of 95.5% and 76.0%, respectively, with Qwen2.5-7B-Instruct. Critic-depth comparisons further show strong task performance with substantially lower critic-side computation. These results suggest that a compact shared critic can coordinate turn- and token-level credit to improve agent performance while keeping value estimation efficient. Code is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.08402 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T11:19:24.829Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).