Metraipolis › archive › dc_7409b8cfd7d3

Guide, Then Let Go: Gap-Adaptive Teacher Scheduling for Sparse-Reward Agentic RL — arXiv:2609.37898

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.37898
License (as archived)
CC0-1.0
Archived
2026-10-07T16:24:15.516Z
SHA-256 of the archived text
8e43de56628a6a1eabbb71f3a49c3553635d112d92777889ea87e28ece57ba87
Stable URL of this page
https://metraipolis.com/documento/dc_7409b8cfd7d3

How to cite

Guide, Then Let Go: Gap-Adaptive Teacher Scheduling for Sparse-Reward Agentic RL — arXiv:2609.37898 — arXiv (en), https://arxiv.org/abs/2609.37898 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7409b8cfd7d3 (Metraipolis archive dc_7409b8cfd7d3, sha256 8e43de56628a6a1eabbb71f3a49c3553635d112d92777889ea87e28ece57ba87)
BibTeX
@misc{dc7409b8cfd7d3,
  title = {Guide, Then Let Go: Gap-Adaptive Teacher Scheduling for Sparse-Reward Agentic RL — arXiv:2609.37898},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.37898},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7409b8cfd7d3; sha256 8e43de56628a6a1eabbb71f3a49c3553635d112d92777889ea87e28ece57ba87},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.37898. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Reinforcement learning for long-horizon agents typically relies on sparse outcome-based rewards. This leads to a severe cold-start problem, as early-stage policies often fail to solve sampled tasks, leaving little useful reward signal for learning. To mitigate this problem, we use on-policy distillation (OPD) to provide token-level guidance on the student's own rollouts. We find that the benefit of this guidance depends on the performance gap between the teacher and the student. When the teacher substantially outperforms the student, distillation helps guide the student through the early training stage where outcome rewards provide little learning signal. As the gap narrows and eventually reverses, however, continued distillation becomes less beneficial and may hinder further improvement. Motivated by this observation, we propose Gap-Adaptive Teacher Scheduling (GATS), which augments the student's RL objective with an OPD term whose weight adapts to the teacher-student performance gap. Specifically, GATS gradually reduces teacher guidance as the student approaches the teacher's reference performance and withdraws it once that reference is reached. This enables GATS to leverage task-trained teachers smaller than the student, since teacher guidance is primarily needed during early training. Across ALFWorld, WebShop, and ScienceWorld with three Qwen2.5 teacher-student configurations, GATS achieves the highest average success rate among the compared methods in all three configurations, improving over reward-only GRPO by 4.37%-11.87% under matched student rollout budgets. Code is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.37898 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T15:54:53.666Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).