Metraipolis › archive › dc_e2ef91daaa65

IAPO: Influence-Aware Policy Optimization for Credit Assignment in Multi-Turn Service Agents — arXiv:2608.24588

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.24588
License (as archived): CC0-1.0
Archived: 2026-10-18T06:36:18.942Z
SHA-256 of the archived text: 6d6ad4c480d10a933e9007aae977aa3e6f28cbdc6867956a791da9de972d367e
Stable URL of this page: https://metraipolis.com/documento/dc_e2ef91daaa65

How to cite

IAPO: Influence-Aware Policy Optimization for Credit Assignment in Multi-Turn Service Agents — arXiv:2608.24588 — arXiv (en), https://arxiv.org/abs/2608.24588 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e2ef91daaa65 (Metraipolis archive dc_e2ef91daaa65, sha256 6d6ad4c480d10a933e9007aae977aa3e6f28cbdc6867956a791da9de972d367e)

BibTeX

@misc{dce2ef91daaa65,
  title = {IAPO: Influence-Aware Policy Optimization for Credit Assignment in Multi-Turn Service Agents — arXiv:2608.24588},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.24588},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e2ef91daaa65; sha256 6d6ad4c480d10a933e9007aae977aa3e6f28cbdc6867956a791da9de972d367e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.24588. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large Language Model (LLM) agents increasingly solve long-horizon tasks through multi-turn interactions with users and external tools. In these settings, relevant task information often unfolds over time rather than being fully specified at the initial prompt. Service agents make this challenge especially concrete: users may clarify or revise their goals, while tool responses provide information needed for subsequent decisions. Thus, a final reward alone cannot indicate which actions contributed to resolving the task. Recent methods rely on comparative evidence from other trajectories or resampled continuations, or on separately constructed step-level learning signals, to refine credit. However, a completed rollout already records how information and errors flow between agent actions. We introduce Influence-Aware Policy Optimization (IAPO), which represents each rollout as a typed influence-dependency graph over trainable agent actions, with user and tool observations serving as evidence. IAPO converts support-use and failed-use structure into routing weights that redistribute the same trajectory-level advantage. Experiments with Qwen3-4B and Qwen3-8B demonstrate superior performance over multi-turn reinforcement learning (RL) baselines across three service-agent benchmarks: ${\tau^2}$-Bench, UserBench, and AgentChangeBench. BFCL-v4 Multi-Turn further shows that these gains do not compromise multi-turn function-calling performance. This work advances the understanding of credit assignment in multi-turn user interactions and provides a principled approach to training service agents from sparse outcome feedback.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-26 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.24588 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T06:28:52.261Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-26?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).