Metraipolis › archive › dc_dca0ba5cf375
Structured Residual Connectivity Matters for Diffusion Transformers — arXiv:2609.33203
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.33203
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T23:51:27.075Z
- SHA-256 of the archived text
94dc271cc06007f47cc0087dbc028ff1a890ab6ea00f7e182496b7d18fa23d6a- Stable URL of this page
https://metraipolis.com/documento/dc_dca0ba5cf375
How to cite
Structured Residual Connectivity Matters for Diffusion Transformers — arXiv:2609.33203 — arXiv (en), https://arxiv.org/abs/2609.33203 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dca0ba5cf375 (Metraipolis archive dc_dca0ba5cf375, sha256 94dc271cc06007f47cc0087dbc028ff1a890ab6ea00f7e182496b7d18fa23d6a)
BibTeX
@misc{dcdca0ba5cf375,
title = {Structured Residual Connectivity Matters for Diffusion Transformers — arXiv:2609.33203},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.33203},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dca0ba5cf375; sha256 94dc271cc06007f47cc0087dbc028ff1a890ab6ea00f7e182496b7d18fa23d6a},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33203. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Diffusion Transformers (DiTs) have established themselves as a scalable backbone for high-fidelity image synthesis. However, unlike U-Net based diffusion models that rely on rigid, hand-crafted skip connections, DiTs predominantly use a uniform residual stream that integrates all preceding layers as a monolithic state. In this work, we rethink residual connections in diffusion transformers and propose to transform them from passive summation into an active retrieval mechanism optimized for image denoising. First, we conduct a systematic analysis of DiT's internal representation, revealing a latent preference for early-layer feature reuse and symmetric layer guidance. Motivated by this, we introduce a structured connectivity design that explicitly integrates local residual connections with long-range pathways. Instead of static skip connections or dense all-layer routing, our method enables each transformer block to selectively ``attend'' to critical earlier representations, dynamically retrieving spatial and semantic cues through direct, differentiable cross-depth paths. Experiments show that our adaptive connectivity leads to faster convergence, with up to $1.73\times$ fewer training iterations, and significant gains in FID and visual quality with less than $0.1\%$ additional parameters, further improving a strong REPA-XL/2 model from $5.9$ to $4.34$ FID without guidance and reaching $1.39$ FID with classifier-free guidance. Our findings suggest that adaptive cross-layer connectivity is a critical yet underexplored factor in diffusion transformers, and that incorporating structured information pathways provides a simple and effective direction for improving scalable generative models. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-29 in cs.CV). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2609.33203 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T23:29:27.163Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).