Metraipolis › archive › dc_43b6a2f640d3
LiSA: Linear Stochastic Attention for Long-Range Visual Modeling — arXiv:2611.04216
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.04216
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-04T12:56:09.279Z
- SHA-256 of the archived text
8acbddb159b348317d8eea86d03d4562e8df98e16c9b99d065b953556116914f- Stable URL of this page
https://metraipolis.com/documento/dc_43b6a2f640d3
How to cite
LiSA: Linear Stochastic Attention for Long-Range Visual Modeling — arXiv:2611.04216 — arXiv (en), https://arxiv.org/abs/2611.04216 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_43b6a2f640d3 (Metraipolis archive dc_43b6a2f640d3, sha256 8acbddb159b348317d8eea86d03d4562e8df98e16c9b99d065b953556116914f)
BibTeX
@misc{dc43b6a2f640d3,
title = {LiSA: Linear Stochastic Attention for Long-Range Visual Modeling — arXiv:2611.04216},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.04216},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_43b6a2f640d3; sha256 8acbddb159b348317d8eea86d03d4562e8df98e16c9b99d065b953556116914f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.04216. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Self-attention has become the standard for modern sequence modeling, enabling Transformers to model long-range dependencies across a wide range of modalities. Despite these advances, its quadratic cost with respect to sequence length remains a significant computational bottleneck. Recent work addresses this limitation by introducing linear variants of self-attention, namely with kernel approximations of the softmax operator. However, their first-order aggregation restricts expressivity and struggles to capture higher-order structure. We propose Linear Stochastic Attention (LiSA), a linear attention plug-in module that enables higher-order token interactions via a stochastic diffusion process based on truncated random walks. LiSA incorporates adaptive locality, favoring semantically similar neighbors to preserve fine-grained structures, while maintaining stable and efficient training. We conduct extensive experiments on classification, semantic segmentation, and object detection tasks and show improved results over state-of-the-art methods. Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.CV, nuovi lavori del giorno). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2611.04216 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-04T12:33:47.109Z dalla pagina https://arxiv.org/list/cs.CV/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).