Metraipolis › archive › dc_d7e4cd73df12
Progress-Aware Distillation for Mitigating Stagnation in Small Language Model Agents — arXiv:2610.19439
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.19439
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T14:15:35.867Z
- SHA-256 of the archived text
9ab762ef80f1b2395d02d44ac94920e2c73efb23d5219dff03f5d889b164ac2d- Stable URL of this page
https://metraipolis.com/documento/dc_d7e4cd73df12
How to cite
Progress-Aware Distillation for Mitigating Stagnation in Small Language Model Agents — arXiv:2610.19439 — arXiv (en), https://arxiv.org/abs/2610.19439 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d7e4cd73df12 (Metraipolis archive dc_d7e4cd73df12, sha256 9ab762ef80f1b2395d02d44ac94920e2c73efb23d5219dff03f5d889b164ac2d)
BibTeX
@misc{dcd7e4cd73df12,
title = {Progress-Aware Distillation for Mitigating Stagnation in Small Language Model Agents — arXiv:2610.19439},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.19439},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d7e4cd73df12; sha256 9ab762ef80f1b2395d02d44ac94920e2c73efb23d5219dff03f5d889b164ac2d},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.19439. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
While LLM-based agent systems have demonstrated remarkable proficiency in reasoning and solving complex real-world tasks, they also come with substantial token consumption and inference latency, limiting their practical deployment. To enhance the efficiency of agent systems, recent efforts have focused on transferring agentic capabilities from large-scale models to small language models (SLMs) via supervised fine-tuning-based distillation. Nevertheless, as interaction trajectories grow longer, involving extended reasoning-action chains and extensive environmental feedback, distilled student agents tend to experience progress stagnation: becoming trapped in unproductive loops characterized by persistent failed strategies and difficulty making meaningful progress. Our systematic analysis shows that this phenomenon consistently occurs across diverse SLMs and task domains. To address this issue, we propose Progress-Aware Distillation (ProD), which explicitly detects and penalizes stagnation in student-generated trajectories. ProD encodes the stagnation gap between student and teacher into an adaptive margin for dynamically penalizing possible stagnation. By iteratively applying this procedure, ProD enables student agents to progressively approximate teacher behavior. Extensive experiments involving 8 SLMs, ranging from 0.6B to 8B parameters, on both in-domain and out-of-domain benchmarks, demonstrate that ProD substantially mitigates progress stagnation while enhancing both the success rates and task-completion efficiency of distilled SLM agents. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.19439 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T13:53:35.600Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).