Metraipolis › archive › dc_2e3a4a8a233a

Hesitation-Aware On-Policy Distillation for Diffusion Language Models — arXiv:2609.33301

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.33301
License (as archived)
CC0-1.0
Archived
2026-10-07T21:01:15.313Z
SHA-256 of the archived text
9d3b065399238e97e5cea154552ce47ecb8fc8bb819f84dcd2a3a95c65f21da5
Stable URL of this page
https://metraipolis.com/documento/dc_2e3a4a8a233a

How to cite

Hesitation-Aware On-Policy Distillation for Diffusion Language Models — arXiv:2609.33301 — arXiv (en), https://arxiv.org/abs/2609.33301 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2e3a4a8a233a (Metraipolis archive dc_2e3a4a8a233a, sha256 9d3b065399238e97e5cea154552ce47ecb8fc8bb819f84dcd2a3a95c65f21da5)
BibTeX
@misc{dc2e3a4a8a233a,
  title = {Hesitation-Aware On-Policy Distillation for Diffusion Language Models — arXiv:2609.33301},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.33301},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2e3a4a8a233a; sha256 9d3b065399238e97e5cea154552ce47ecb8fc8bb819f84dcd2a3a95c65f21da5},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33301. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Diffusion large language models (dLLMs) generate text by iterative unmasking. At each denoising step, a dLLM proposes a token at every masked position, but the decoder commits only a confident subset of these proposals. Trace-based on-policy distillation (TOPD) builds on this process by matching the student to a stronger teacher, yet only at the committed positions. We argue that this discards much of the useful signal, which resides in the uncommitted proposals, where the student has made a prediction but is not yet confident enough to commit it. We call these proposals hesitations. In our pilot study on an SDAR-4B student, hesitations make up only 24% of supervisable state-position pairs but carry 66% of the teacher-student divergence. To exploit this signal, we propose Hesitation-Aware On-Policy Distillation (HOPD), which extends teacher distribution matching to every masked position of each denoising step. Because hesitations are not equally informative, we further allocate supervision using hindsight from the completed trajectory, placing more weight on positions whose proposal was later disagreed with the final token and on blocks where first-step proposals rarely survive. Since both models already produce distributions at all masked positions, HOPD requires no additional forward passes over TOPD. The only extra cost is evaluating the loss at more positions. With SDAR-1.7B and SDAR-4B students distilled from TraDo-8B-Instruct, HOPD achieves the best average score among the evaluated methods on five math and coding benchmarks, under both static and dynamic decoding and at both scales. It also speeds up decoding. On SDAR-4B, the HOPD student hesitates less and commits 11% more tokens per denoising step than TOPD, while reaching higher accuracy.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.33301 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:29:02.603Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).