Metraipolis › archive › dc_4fab79197d4e

Poly-OPD: Heterogeneous Multi-Teacher On-Policy Distillation for Capability-Selectable Flow Models — arXiv:2608.04349

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.04349
License (as archived)
CC0-1.0
Archived
2026-10-31T11:47:49.686Z
SHA-256 of the archived text
e8e32cb5c030735e3556fe5f0193bc2db543c48788c59470d398c439d9d0a0e9
Stable URL of this page
https://metraipolis.com/documento/dc_4fab79197d4e

How to cite

Poly-OPD: Heterogeneous Multi-Teacher On-Policy Distillation for Capability-Selectable Flow Models — arXiv:2608.04349 — arXiv (en), https://arxiv.org/abs/2608.04349 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4fab79197d4e (Metraipolis archive dc_4fab79197d4e, sha256 e8e32cb5c030735e3556fe5f0193bc2db543c48788c59470d398c439d9d0a0e9)
BibTeX
@misc{dc4fab79197d4e,
  title = {Poly-OPD: Heterogeneous Multi-Teacher On-Policy Distillation for Capability-Selectable Flow Models — arXiv:2608.04349},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.04349},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4fab79197d4e; sha256 e8e32cb5c030735e3556fe5f0193bc2db543c48788c59470d398c439d9d0a0e9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.04349. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Leading open text-to-image models often carry complementary strengths: one may lead on preference-aligned aesthetics while another follows compositional instructions more faithfully. However, differences in their autoencoders and noise schedules make it difficult to transfer these strengths across models. In this paper, we present Poly-OPD, a framework that can consolidate complementary strengths of heterogeneous teachers into a single compact flow-matching student. To bridge the incompatible latent spaces of different teachers, Poly-OPD performs on-policy distillation through a pixel bridge. Each student-generated image is re-encoded by a selected teacher's encoder and refined from a noise level matched by magnitude under the teacher's noise schedule. The resulting target is further matched to the student in frozen DINOv2 space, enabling supervision across incompatible latent spaces. To retain complementary capabilities without cross-teacher interference, Poly-OPD uses a gradient compatibility diagnostic to organize its adapters: attention LoRA modules are shared across teachers, whereas feed-forward adapters remain teacher-specific. During distillation, a gap-aware curriculum devotes more training to compositional categories where the student still falls short of the teacher. As each gap narrows, training shifts toward categories with larger remaining gaps. By distilling FLUX.1-dev and Z-Image into a 2.5B SD3.5-Medium student, Poly-OPD improves GenEval from 67.3 to 73.3, surpassing both larger teachers, and raises DrawBench HPSv3 from 9.34 to 11.35, consolidating both strengths within a switchable model.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-06 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2608.04349 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T11:37:40.656Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-08-06?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).