Metraipolis › archive › dc_4fab79197d4e
Poly-OPD: Heterogeneous Multi-Teacher On-Policy Distillation for Capability-Selectable Flow Models — arXiv:2608.04349
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.04349
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-31T11:47:49.686Z
- SHA-256 of the archived text
e8e32cb5c030735e3556fe5f0193bc2db543c48788c59470d398c439d9d0a0e9- Stable URL of this page
https://metraipolis.com/documento/dc_4fab79197d4e
How to cite
Poly-OPD: Heterogeneous Multi-Teacher On-Policy Distillation for Capability-Selectable Flow Models — arXiv:2608.04349 — arXiv (en), https://arxiv.org/abs/2608.04349 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4fab79197d4e (Metraipolis archive dc_4fab79197d4e, sha256 e8e32cb5c030735e3556fe5f0193bc2db543c48788c59470d398c439d9d0a0e9)
BibTeX
@misc{dc4fab79197d4e,
title = {Poly-OPD: Heterogeneous Multi-Teacher On-Policy Distillation for Capability-Selectable Flow Models — arXiv:2608.04349},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.04349},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4fab79197d4e; sha256 e8e32cb5c030735e3556fe5f0193bc2db543c48788c59470d398c439d9d0a0e9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.04349. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Leading open text-to-image models often carry complementary strengths: one may lead on preference-aligned aesthetics while another follows compositional instructions more faithfully. However, differences in their autoencoders and noise schedules make it difficult to transfer these strengths across models. In this paper, we present Poly-OPD, a framework that can consolidate complementary strengths of heterogeneous teachers into a single compact flow-matching student. To bridge the incompatible latent spaces of different teachers, Poly-OPD performs on-policy distillation through a pixel bridge. Each student-generated image is re-encoded by a selected teacher's encoder and refined from a noise level matched by magnitude under the teacher's noise schedule. The resulting target is further matched to the student in frozen DINOv2 space, enabling supervision across incompatible latent spaces. To retain complementary capabilities without cross-teacher interference, Poly-OPD uses a gradient compatibility diagnostic to organize its adapters: attention LoRA modules are shared across teachers, whereas feed-forward adapters remain teacher-specific. During distillation, a gap-aware curriculum devotes more training to compositional categories where the student still falls short of the teacher. As each gap narrows, training shifts toward categories with larger remaining gaps. By distilling FLUX.1-dev and Z-Image into a 2.5B SD3.5-Medium student, Poly-OPD improves GenEval from 67.3 to 73.3, surpassing both larger teachers, and raises DrawBench HPSv3 from 9.34 to 11.35, consolidating both strengths within a switchable model. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-06 in cs.CV). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2608.04349 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-31T11:37:40.656Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-08-06?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).