Metraipolis › archive › dc_7af01a600c91
More Value per Key: Asymmetric Sparse Attention for Faster LLM Decoding — arXiv:2610.04753
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.04753
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T05:26:25.116Z
- SHA-256 of the archived text
713e20eac2c39c9cb4ad350c652be040aedd3dbda7816155585c0178ae4f3963- Stable URL of this page
https://metraipolis.com/documento/dc_7af01a600c91
How to cite
More Value per Key: Asymmetric Sparse Attention for Faster LLM Decoding — arXiv:2610.04753 — arXiv, https://arxiv.org/abs/2610.04753 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7af01a600c91 (Metraipolis archive dc_7af01a600c91, sha256 713e20eac2c39c9cb4ad350c652be040aedd3dbda7816155585c0178ae4f3963)
BibTeX
@misc{dc7af01a600c91,
title = {More Value per Key: Asymmetric Sparse Attention for Faster LLM Decoding — arXiv:2610.04753},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.04753},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7af01a600c91; sha256 713e20eac2c39c9cb4ad350c652be040aedd3dbda7816155585c0178ae4f3963},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.04753. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
utoregressive generation in Large Language Models (LLMs) is constrained by the memory and computational demands of attention mechanisms. Sparse attention methods mitigate this cost by selecting only high-probability entries of the attention matrix. We observe that in many such methods, this renders the probability-value multiplication negligible, shifting the bottleneck to the query-key step. Key heads can therefore be reduced to accelerate inference, while retaining more value heads preserves capacity with limited additional decoding cost. We introduce Sparse Asymmetric Group-Query Attention (SAGA), which decouples key and value head counts to exploit this principle, and pair it with approximate top-N (Atop-N) attention, a simple sparse attention method designed to study the interaction between sparsity and head-count asymmetry. We formalize the benefits of this asymmetry theoretically and validate them empirically through latency measurements and quality evaluations on models up to 1.5B parameters. Together, SAGA and Atop-N achieve end-to-end decoding speedups exceeding $2\times$ over our full-attention GQA baseline at long contexts. Models trained from scratch with SAGA nearly match the quality of comparable GQA variants on the evaluated benchmarks. To facilitate adoption, we introduce an efficient fine-tuning method that converts pretrained models to the SAGA architecture, enabling practitioners to benefit from our approach without costly retraining. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.04753 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-06T05:24:05.910Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).