Metraipolis › archive › dc_b3436b9bbd2c
Agentic-OPD: Recovering Forgotten Agent Capabilities via On-Policy Distillation for Agentic Tasks — arXiv:2611.12799
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.12799
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-10T18:44:17.794Z
- SHA-256 of the archived text
9740c8da27b1f268370951a50ea25907b1077d0bd28a0f58d262c88bc2d2a317- Stable URL of this page
https://metraipolis.com/documento/dc_b3436b9bbd2c
How to cite
Agentic-OPD: Recovering Forgotten Agent Capabilities via On-Policy Distillation for Agentic Tasks — arXiv:2611.12799 — arXiv (en), https://arxiv.org/abs/2611.12799 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b3436b9bbd2c (Metraipolis archive dc_b3436b9bbd2c, sha256 9740c8da27b1f268370951a50ea25907b1077d0bd28a0f58d262c88bc2d2a317)
BibTeX
@misc{dcb3436b9bbd2c,
title = {Agentic-OPD: Recovering Forgotten Agent Capabilities via On-Policy Distillation for Agentic Tasks — arXiv:2611.12799},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.12799},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b3436b9bbd2c; sha256 9740c8da27b1f268370951a50ea25907b1077d0bd28a0f58d262c88bc2d2a317},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.12799. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Reinforcement learning is now the mainstream paradigm for training LLMs on agentic tasks, yet standard practice keeps only the best-validation checkpoint and discards the rest. We show that this is lossy: in GRPO training of LLM agents, the best-validation checkpoint fails on problems that earlier checkpoints from the same run had already solved, and the union of problems solved across checkpoints far exceeds any single one. The discarded checkpoints thus form a teacher pool obtained at no additional training cost, which on-policy distillation (OPD) can mine to recover the lost coverage. Mining this pool raises two challenges: i) which checkpoint to distill from, as no checkpoint dominates the others and complementarity to the student is problem-specific; ii) where to trust it, as even a well-chosen teacher is only locally superior, so indiscriminate distillation overwrites behavior the student has already mastered. We propose Agentic-OPD, which treats the best-validation checkpoint as the student and all saved checkpoints as frozen teacher candidates, using the student's evolving capability gaps to guide both teacher selection and supervision filtering: the HCN policy (Highest Coverage Next) dynamically selects teachers by weighted coverage of the student's unsolved problems, while the LAD module (Locality-Aware Distillation) confines transfer to failed student rollouts on problems where the teacher performs better. Across five agentic task domains and 24 benchmarks and task subsets, Agentic-OPD improves over its initialization checkpoint by up to 7.4 points, with a 4B student surpassing the latest frontier models GPT-5.6 and DeepSeek-V4.1 on deep search. Agentic-OPD recasts a finished RL run as a reusable pool of policies and establishes post-RL capability recovery as a stage orthogonal to improvements in the RL algorithm itself. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2611.12799 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-10T18:37:37.707Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).