Metraipolis › archive › dc_052d1bf296da

FocusOPD: Structure-Guided On-Policy Distillation for Diffusion Models — arXiv:2610.21250

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.21250
License (as archived): CC0-1.0
Archived: 2026-10-16T10:35:13.449Z
SHA-256 of the archived text: a0ef7b1dbc4acafd8bba51bb2ab746e8ecfe5d2544aaee277fcf2131ba957908
Stable URL of this page: https://metraipolis.com/documento/dc_052d1bf296da

How to cite

FocusOPD: Structure-Guided On-Policy Distillation for Diffusion Models — arXiv:2610.21250 — arXiv (en), https://arxiv.org/abs/2610.21250 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_052d1bf296da (Metraipolis archive dc_052d1bf296da, sha256 a0ef7b1dbc4acafd8bba51bb2ab746e8ecfe5d2544aaee277fcf2131ba957908)

BibTeX

@misc{dc052d1bf296da,
  title = {FocusOPD: Structure-Guided On-Policy Distillation for Diffusion Models — arXiv:2610.21250},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.21250},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_052d1bf296da; sha256 a0ef7b1dbc4acafd8bba51bb2ab746e8ecfe5d2544aaee277fcf2131ba957908},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.21250. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multi-teacher on-policy distillation enables rectified-flow models with diffusion-transformer architectures to combine task-specific capabilities within a single student. However, matching model outputs does not guarantee that their internal representations are aligned, and it remains unclear where and when additional feature supervision is most useful. We analyze the teacher-student feature mismatch across Transformer blocks and denoising timesteps, and find that the mismatch is concentrated in deeper Transformer blocks during early denoising and persists after OPD distillation, indicating ineffective distillation. Motivated by this observation, we introduce FocusOPD, which augments the base OPD objective with token-wise directional feature alignment restricted to deep layers and early denoising steps. Across two rectified-flow backbones and multiple OPD objectives, FocusOPD improves capability transfer in compositional generation, text rendering, and preference-based evaluation. On the primary benchmark, it improves over DiffusionOPD from 0.953 to 0.967 on GenEval, from 23.797 to 23.944 on PickScore, and from 6.052 to 6.199 on LAION-Aesthetic. Ablations show that selective deep and early supervision is more effective and efficient than alignment over the full network and trajectory.

Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.CV, nuovi lavori del giorno).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2610.21250 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T08:52:02.644Z dalla pagina https://arxiv.org/list/cs.CV/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).