Metraipolis › archive › dc_0b61394ab2b4

HiDiffTIR: Hierarchical Difficulty-Aware Policy Optimization for Multi-Turn Tool-Integrated Reasoning — arXiv:2608.21863

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.21863
License (as archived)
CC0-1.0
Archived
2026-10-18T07:42:28.311Z
SHA-256 of the archived text
90265d9e02079aff171f6ac8e52dd0a328c8fb4feb5ee4b61b86ddde410c5d11
Stable URL of this page
https://metraipolis.com/documento/dc_0b61394ab2b4

How to cite

HiDiffTIR: Hierarchical Difficulty-Aware Policy Optimization for Multi-Turn Tool-Integrated Reasoning — arXiv:2608.21863 — arXiv (en), https://arxiv.org/abs/2608.21863 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0b61394ab2b4 (Metraipolis archive dc_0b61394ab2b4, sha256 90265d9e02079aff171f6ac8e52dd0a328c8fb4feb5ee4b61b86ddde410c5d11)
BibTeX
@misc{dc0b61394ab2b4,
  title = {HiDiffTIR: Hierarchical Difficulty-Aware Policy Optimization for Multi-Turn Tool-Integrated Reasoning — arXiv:2608.21863},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.21863},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0b61394ab2b4; sha256 90265d9e02079aff171f6ac8e52dd0a328c8fb4feb5ee4b61b86ddde410c5d11},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.21863. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Tool-Integrated Reasoning (TIR) is a fundamental capability for LLM agents to solve complex tasks by interacting with external tools iteratively. Reinforcement Learning (RL) has become the dominant paradigm for enabling this capability. However, existing approaches typically assign uniform trajectory-level advantages and treat all correct tool calls equally, ignoring the varying difficulty and learning value across trajectories and reasoning steps. This can lead to imprecise learning signals that do not adequately distinguish between trivial and challenging tool-use patterns. To address this limitation, we propose HiDiffTIR, a Hierarchical Difficulty-aware policy optimization framework for multi-turn TIR. HiDiffTIR performs difficulty-aware credit assignment at both trajectory and turn levels, enabling the policy to focus on more informative trajectories and harder reasoning steps. Notably, this fine-grained optimization is achieved without additional supervision, relying solely on group-level statistics derived from standard RL rollouts. Extensive experiments on three tool-using benchmarks demonstrate that HiDiffTIR consistently improves multi-turn TIR performance and tool invocation accuracy over strong RL baselines, highlighting the necessity of difficulty-aware credit assignment for effective policy optimization in tool-integrated LLM agents.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-25 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.21863 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T07:37:52.837Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).