Metraipolis › archive › dc_d67e8d39bdd2
What Visual Generators Need from Teachers: Rethinking Representation Alignment — arXiv:2609.34732
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.34732
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T23:52:24.013Z
- SHA-256 of the archived text
a5b39c2cf41c349b8ee0fd278db1841046e7b03dd5d0a23b4352ddb4f8227924- Stable URL of this page
https://metraipolis.com/documento/dc_d67e8d39bdd2
How to cite
What Visual Generators Need from Teachers: Rethinking Representation Alignment — arXiv:2609.34732 — arXiv (en), https://arxiv.org/abs/2609.34732 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d67e8d39bdd2 (Metraipolis archive dc_d67e8d39bdd2, sha256 a5b39c2cf41c349b8ee0fd278db1841046e7b03dd5d0a23b4352ddb4f8227924)
BibTeX
@misc{dcd67e8d39bdd2,
title = {What Visual Generators Need from Teachers: Rethinking Representation Alignment — arXiv:2609.34732},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.34732},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d67e8d39bdd2; sha256 a5b39c2cf41c349b8ee0fd278db1841046e7b03dd5d0a23b4352ddb4f8227924},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.34732. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Representation alignment speeds up diffusion transformer training by pulling an intermediate block of the model (student) toward features of a frozen pretrained encoder (teacher). Which teacher layer to align, and for how long, is still set by convention, and each alternative costs a training run. We find that alignment helps where the student cannot linearly recover the teacher's features, not where it already resembles them. Since a deep teacher layer is largely predictable from the one below, we isolate what each layer adds, its increment, and measure how much of it an unaligned student recovers. The student fills the teacher's hierarchy from the bottom up and stalls near the top, which we call hierarchy filling: even after 400K steps it recovers almost none of the deepest. The recoverability gap is the unrecovered share of an increment, read from one unaligned checkpoint. In short runs that each align one teacher layer at one block, the gap nearly reproduces their ranking by FID improvement, and CKA, a measure of feature similarity, largely reverses it. Representation Alignment and Recoverability Estimation (RARE) picks the teacher layer with the largest gap before training. During training, it tracks each token's remaining distance to that layer, the online counterpart of the gap, weights tokens by it, and phases out the loss once the average distance stops falling. With SiT-B/2 on ImageNet $256\times256$, RARE reaches an FID of 18.02 without guidance and 4.46 with it, ahead of seven alignment baselines including REPA, iREPA and HASTE. It also trains in 14% fewer GPU-hours than iREPA. Its FID stays below iREPA's across model scales, teachers, datasets and backbones. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-29 in cs.CV). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2609.34732 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T23:29:27.442Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).