Metraipolis › archive › dc_04f04529583a
No Channels Left Behind: Understanding Diffusion Acceleration through Residual Gates — arXiv:2610.40914
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.40914
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T05:40:37.218Z
- SHA-256 of the archived text
5d50d5c6925c80752d6589c86155216c79babbbf892d4b27c194e030cd85dce6- Stable URL of this page
https://metraipolis.com/documento/dc_04f04529583a
How to cite
No Channels Left Behind: Understanding Diffusion Acceleration through Residual Gates — arXiv:2610.40914 — arXiv (en), https://arxiv.org/abs/2610.40914 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_04f04529583a (Metraipolis archive dc_04f04529583a, sha256 5d50d5c6925c80752d6589c86155216c79babbbf892d4b27c194e030cd85dce6)
BibTeX
@misc{dc04f04529583a,
title = {No Channels Left Behind: Understanding Diffusion Acceleration through Residual Gates — arXiv:2610.40914},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.40914},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_04f04529583a; sha256 5d50d5c6925c80752d6589c86155216c79babbbf892d4b27c194e030cd85dce6},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.40914. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Training acceleration methods for diffusion transformers, such as REPA and dispersive regularization, are commonly explained by additional supervision, spatial-structure priors, or improved feature geometry. We revisit this view and uncover a striking internal change: the AdaLN-Zero residual gate $\alpha$, which directly controls how conditioned updates enter the residual stream, is substantially reorganized by these methods. This observation motivates our hypothesis that acceleration arises, at least in part, from reshaping the residual gating pathway rather than solely from the feature structures explicitly imposed by auxiliary objectives. Supporting this view, accelerated models are markedly more sensitive to perturbations of $\alpha$, and their gate geometry reflects semantic relationships among classes. We further introduce the normalized participation ratio (nPR) to characterize the channel-wise organization of the gates and find that, across REPA teachers, projector architectures, and radial-objective targets, broader gate participation accompanies better generation quality. Guided by this insight, we propose a simple teacher-free radial objective that promotes broader gate participation through feature-norm regularization, without pairwise feature comparisons. On ImageNet $256\times256$ with SiT-B/2, our method improves FID over dispersive loss while increasing mean gate participation over timesteps and classes, further supporting the link between gate organization and training acceleration. Our results identify residual-gate reorganization as a measurable axis of diffusion training acceleration and suggest a simple direction for further acceleration. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2610.40914 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T05:26:24.757Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).