Metraipolis › archive › dc_85d9c6744ff3
GFD-OPD: Guidance-Folded On-Policy Distillation of Diffusion Models Across Scales — arXiv:2609.39692
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.39692
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T12:45:42.800Z
- SHA-256 of the archived text
bbca606ac50dc07169b18814e16b674ea84fb8c09844c028a3db217b9dc2a502- Stable URL of this page
https://metraipolis.com/documento/dc_85d9c6744ff3
How to cite
GFD-OPD: Guidance-Folded On-Policy Distillation of Diffusion Models Across Scales — arXiv:2609.39692 — arXiv (en), https://arxiv.org/abs/2609.39692 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_85d9c6744ff3 (Metraipolis archive dc_85d9c6744ff3, sha256 bbca606ac50dc07169b18814e16b674ea84fb8c09844c028a3db217b9dc2a502)
BibTeX
@misc{dc85d9c6744ff3,
title = {GFD-OPD: Guidance-Folded On-Policy Distillation of Diffusion Models Across Scales — arXiv:2609.39692},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.39692},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_85d9c6744ff3; sha256 bbca606ac50dc07169b18814e16b674ea84fb8c09844c028a3db217b9dc2a502},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.39692. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
On-policy distillation (OPD) has demonstrated two important capabilities in language models: compressing large teachers into smaller students and merging expert models into a single model. Existing diffusion OPD, however, mostly focus on the latter, with teachers and students sharing the same backbone and scale. We investigate large-to-small diffusion opd from large teachers to a small student and find that the standard recipe fails. To find the underlying cause, we propose Fixed-State KL, an effective and fair way to measure the distribution gap between student and teacher during OPD training for diffusion models. We are the first to clarify why large-to-small OPD is challenging for diffusion models: a smaller student struggles to perfectly match the distribution of a larger teacher, while classifier-free guidance can accumulate and amplify the distributional discrepancies between the student's conditional and unconditional branches and those of the teacher. To solve this problem, we propose GFD-OPD, a simple yet effective method that reduces the student-teacher gap while avoiding the error amplification of the CFG composition. Across numerous experiments, GFD outperforms previous baselines in both training efficiency and final performance, achieving state-of-the-art results on all benchmarks. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in cs.AI). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.39692 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T12:14:14.139Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).