Metraipolis › archive › dc_cea58e6cd68f
Backend-Agnostic Sparse Attention for Fast High-Resolution Visual Generation — arXiv:2610.08772
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.08772
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T13:07:52.425Z
- SHA-256 of the archived text
665977da7c697f56be72a605304cb223cbc7d6e4227235015a9aa03fc72876e9- Stable URL of this page
https://metraipolis.com/documento/dc_cea58e6cd68f
How to cite
Backend-Agnostic Sparse Attention for Fast High-Resolution Visual Generation — arXiv:2610.08772 — arXiv (en), https://arxiv.org/abs/2610.08772 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cea58e6cd68f (Metraipolis archive dc_cea58e6cd68f, sha256 665977da7c697f56be72a605304cb223cbc7d6e4227235015a9aa03fc72876e9)
BibTeX
@misc{dccea58e6cd68f,
title = {Backend-Agnostic Sparse Attention for Fast High-Resolution Visual Generation — arXiv:2610.08772},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.08772},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cea58e6cd68f; sha256 665977da7c697f56be72a605304cb223cbc7d6e4227235015a9aa03fc72876e9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.08772. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Diffusion Transformers (DiTs) have achieved strong performance in image and video generation, but the quadratic complexity of full attention makes high-resolution generation computationally expensive. Window attention offers an efficient alternative, yet existing methods face a practical trade-off: partitioned window attention typically achieves computational efficiency consistent with its theoretical complexity. However, isolated windows block cross-window interaction, often introducing visible grid-like artifacts in the generated results. Fine-grained sliding-window attention effectively restores interactions across neighboring windows and improves visual quality. However, its irregular computation patterns create a substantial gap between theoretical and practical speedups and require specialized kernels tailored to each hardware backend. To tackle these challenges, we propose BASA, a backend-agnostic sparse attention, which brings the best of both worlds: visual quality and practical acceleration. Specifically, BASA replaces visual self-attention with shifted local-window attention. By introducing a structured window-shifting scheme across DiT blocks, we allow tokens divided by window boundaries in one layer to communicate in the following layers, thereby achieving global information exchange and eliminating window-induced visual artifacts. Notably, our design introduces no additional irregular operators or customized kernels, making it readily deployable on existing attention backends and closing the gap between theoretical sparsity and practical acceleration. Experiments demonstrate that BASA achieves measured speedups exceeding 90\% of the theoretical estimates on FLUX and delivers a 4.52$\times$ attention speedup on Wan while maintaining competitive generation quality. Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.CV, nuovi lavori del giorno). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2610.08772 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T12:37:33.019Z dalla pagina https://arxiv.org/list/cs.CV/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).