Metraipolis › archive › dc_aa26a3186bd5

TIDE: Teacher-Student Transition via Informative Distillation and Exploration for Agentic RL — arXiv:2609.35058

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.35058
License (as archived)
CC0-1.0
Archived
2026-10-07T21:03:14.797Z
SHA-256 of the archived text
d7c5b3d4f664045ca850259430e854cba39e78ed89ac7d0a024e82db219daf1b
Stable URL of this page
https://metraipolis.com/documento/dc_aa26a3186bd5

How to cite

TIDE: Teacher-Student Transition via Informative Distillation and Exploration for Agentic RL — arXiv:2609.35058 — arXiv (en), https://arxiv.org/abs/2609.35058 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_aa26a3186bd5 (Metraipolis archive dc_aa26a3186bd5, sha256 d7c5b3d4f664045ca850259430e854cba39e78ed89ac7d0a024e82db219daf1b)
BibTeX
@misc{dcaa26a3186bd5,
  title = {TIDE: Teacher-Student Transition via Informative Distillation and Exploration for Agentic RL — arXiv:2609.35058},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.35058},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_aa26a3186bd5; sha256 d7c5b3d4f664045ca850259430e854cba39e78ed89ac7d0a024e82db219daf1b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.35058. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Effective multi-turn agents require interaction strategies that coordinate information gathering, actions, and feedback over long horizons. GRPO is a reinforcement learning algorithm used to train these agents, but sparse trajectory-level rewards limit early exploration in small models. Recent methods augment RL with on-policy distillation (OPD) from a stronger teacher. However, a fixed mixture assumes that teacher guidance and reward optimization should retain a constant relative role throughout training and across interaction turns. This assumption can fail at two scales. Globally, as training progresses, maintaining strong distillation pressure can constrain the model from moving beyond the teacher's capabilities. Locally, teacher--student disagreement identifies where the student departs from the teacher, but cannot tell whether that departure is exploration supported by better outcomes or low-quality policy drift. Our methodological insight is that teacher guidance and reward optimization should be dynamically rebalanced over training and jointly allocated across turns. We instantiate this insight in \tide. Globally, \tide uses the measured disagreement trend as a practical schedule signal, advancing an OPD-to-RL handoff when discrepancy reduction becomes slow but remains positive and progressively increasing the relative weight of RL. Locally, \tide jointly modulates teacher-guided and reward-driven updates: relative action value and disagreement prioritize the OPD signal, whereas relative action value supplies the RL advantage and normalized disagreement reweights it across turns. Coupled with the global handoff, \tide allocates stronger teacher guidance early and gives reward-driven updates greater relative weight later in training. Experiments across multiple benchmarks, student scales, and controlled ablations support the effectiveness of TIDE's adaptive OPD--RL coordination.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.35058 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:29:03.249Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).