Metraipolis › archive › dc_f611fa27bf4b

Dense Is Not Enough: Hierarchical Supervision Allocation for Long-Horizon On-Policy Distillation — arXiv:2609.33409

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.33409
License (as archived)
CC0-1.0
Archived
2026-10-07T21:53:00.852Z
SHA-256 of the archived text
a7fe5889c4209bd8bf2697fc4081aae1d862df32b94dcda83e77246d12bc4954
Stable URL of this page
https://metraipolis.com/documento/dc_f611fa27bf4b

How to cite

Dense Is Not Enough: Hierarchical Supervision Allocation for Long-Horizon On-Policy Distillation — arXiv:2609.33409 — arXiv (en), https://arxiv.org/abs/2609.33409 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f611fa27bf4b (Metraipolis archive dc_f611fa27bf4b, sha256 a7fe5889c4209bd8bf2697fc4081aae1d862df32b94dcda83e77246d12bc4954)
BibTeX
@misc{dcf611fa27bf4b,
  title = {Dense Is Not Enough: Hierarchical Supervision Allocation for Long-Horizon On-Policy Distillation — arXiv:2609.33409},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.33409},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f611fa27bf4b; sha256 a7fe5889c4209bd8bf2697fc4081aae1d862df32b94dcda83e77246d12bc4954},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33409. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

On-policy distillation (OPD) transfers the capabilities of a large language model to a smaller student by providing teacher supervision on the student's own rollouts. In long-horizon agentic tasks, however, uniform token-level matching can allocate supervision poorly: a large local discrepancy need not improve future behavior, while consequential guidance may be beyond the current student's reach or fail to persist without privileged input. We formulate long-horizon OPD as hierarchical supervision allocation and argue that productive guidance lies at the intersection of future utility and current learnability. Crucially, this intersection evolves as the student learns. Based on this principle, we propose LENS-OPD, a coarse-to-fine framework that organizes supervision through Locate, Validate, and Refine. Locate adapts trajectory exposure to the student's evolving competence and proposes a candidate decision for intervention. Validate tests whether teacher guidance at that decision improves the same student's subsequent behavior. Refine internalizes the beneficial guided behavior into the deployable policy and concentrates token-level supervision on decisive teacher-student conflicts within the validated turn. These stages are nested: each finer allocation is conditioned on the coarser decision, rather than being optimized as an independent importance score. Experiments across multiple long-horizon agent benchmarks and student-teacher configurations show that LENS-OPD consistently improves task performance over vanilla OPD and strong curriculum- and selection-based baselines. Our results suggest that effective long-horizon distillation requires teaching at the right depth, the right decision, and the right token.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.33409 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T21:27:08.515Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).