Metraipolis › archive › dc_fa101b6ed81f
Prism Attention: Proposal-Refined Index Sharing Mechanism for Efficient LLMs Inference — arXiv:2610.22985
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.22985
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T08:43:05.371Z
- SHA-256 of the archived text
ca1c426780bb88b5ff004b84235b43173864312f3de050d221641b7c42147278- Stable URL of this page
https://metraipolis.com/documento/dc_fa101b6ed81f
How to cite
Prism Attention: Proposal-Refined Index Sharing Mechanism for Efficient LLMs Inference — arXiv:2610.22985 — arXiv (en), https://arxiv.org/abs/2610.22985 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fa101b6ed81f (Metraipolis archive dc_fa101b6ed81f, sha256 ca1c426780bb88b5ff004b84235b43173864312f3de050d221641b7c42147278)
BibTeX
@misc{dcfa101b6ed81f,
title = {Prism Attention: Proposal-Refined Index Sharing Mechanism for Efficient LLMs Inference — arXiv:2610.22985},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.22985},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fa101b6ed81f; sha256 ca1c426780bb88b5ff004b84235b43173864312f3de050d221641b7c42147278},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.22985. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Long-context decoding in large language models (LLMs) is increasingly bottlenecked by the memory and computation required to attend over an ever-growing Key-Value (KV) cache at each autoregressive step. Although token-level sparse attention reduces this burden by restricting computation to a small subset of relevant past tokens, its practical gains are often offset by repeatedly recomputing layer-wise sparse indices as the context grows. A common remedy is to share sparse indices across layers by computing them only at designated anchor layers, but such static reuse ignores the layer-wise evolution of attention patterns, causing the reused indices to progressively deviate from the layer-specific optima and incur substantial attention loss. By revisiting cross-layer sparsity from the perspective of attention dynamics, we find that inter-layer salient token shift is highly localized: although the exact top-$\kappa$ indices differ across layers, the core high-attention tokens remain stable, with changes concentrated near the expanded boundary of previously selected regions. Motivated by this, we propose Prism Attention, a proposal-refined sparse attention framework. It replaces rigid cross-layer index reuse with a coarse-to-fine strategy, preserving layer-wise adaptability while retaining the efficiency benefits of cross-layer sharing. Extensive experiments show that Prism Attention achieves state-of-the-art performance among mainstream sparse attention methods on both reasoning and long-context benchmarks, while attaining a $3.4\times$ end-to-end decoding speedup over FlashAttention at 128K context length. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.22985 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T08:39:10.137Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).