Metraipolis › archive › dc_fa101b6ed81f

Prism Attention: Proposal-Refined Index Sharing Mechanism for Efficient LLMs Inference — arXiv:2610.22985

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.22985
License (as archived)
CC0-1.0
Archived
2026-10-19T08:43:05.371Z
SHA-256 of the archived text
ca1c426780bb88b5ff004b84235b43173864312f3de050d221641b7c42147278
Stable URL of this page
https://metraipolis.com/documento/dc_fa101b6ed81f

How to cite

Prism Attention: Proposal-Refined Index Sharing Mechanism for Efficient LLMs Inference — arXiv:2610.22985 — arXiv (en), https://arxiv.org/abs/2610.22985 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fa101b6ed81f (Metraipolis archive dc_fa101b6ed81f, sha256 ca1c426780bb88b5ff004b84235b43173864312f3de050d221641b7c42147278)
BibTeX
@misc{dcfa101b6ed81f,
  title = {Prism Attention: Proposal-Refined Index Sharing Mechanism for Efficient LLMs Inference — arXiv:2610.22985},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.22985},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fa101b6ed81f; sha256 ca1c426780bb88b5ff004b84235b43173864312f3de050d221641b7c42147278},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.22985. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Long-context decoding in large language models (LLMs) is increasingly bottlenecked by the memory and computation required to attend over an ever-growing Key-Value (KV) cache at each autoregressive step. Although token-level sparse attention reduces this burden by restricting computation to a small subset of relevant past tokens, its practical gains are often offset by repeatedly recomputing layer-wise sparse indices as the context grows. A common remedy is to share sparse indices across layers by computing them only at designated anchor layers, but such static reuse ignores the layer-wise evolution of attention patterns, causing the reused indices to progressively deviate from the layer-specific optima and incur substantial attention loss. By revisiting cross-layer sparsity from the perspective of attention dynamics, we find that inter-layer salient token shift is highly localized: although the exact top-$\kappa$ indices differ across layers, the core high-attention tokens remain stable, with changes concentrated near the expanded boundary of previously selected regions. Motivated by this, we propose Prism Attention, a proposal-refined sparse attention framework. It replaces rigid cross-layer index reuse with a coarse-to-fine strategy, preserving layer-wise adaptability while retaining the efficiency benefits of cross-layer sharing. Extensive experiments show that Prism Attention achieves state-of-the-art performance among mainstream sparse attention methods on both reasoning and long-context benchmarks, while attaining a $3.4\times$ end-to-end decoding speedup over FlashAttention at 128K context length.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.22985 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T08:39:10.137Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).