Metraipolis › archive › dc_2e66314e63fc
OpenAttention: One Percent Is Enough with Tail Proxies — arXiv:2610.30951
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.30951
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-23T15:15:14.079Z
- SHA-256 of the archived text
454c18f5eba43e82885eb8eca48bbfdd3cdafaa1f6c01ee9940f22b95959fc1f- Stable URL of this page
https://metraipolis.com/documento/dc_2e66314e63fc
How to cite
OpenAttention: One Percent Is Enough with Tail Proxies — arXiv:2610.30951 — arXiv (en), https://arxiv.org/abs/2610.30951 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2e66314e63fc (Metraipolis archive dc_2e66314e63fc, sha256 454c18f5eba43e82885eb8eca48bbfdd3cdafaa1f6c01ee9940f22b95959fc1f)
BibTeX
@misc{dc2e66314e63fc,
title = {OpenAttention: One Percent Is Enough with Tail Proxies — arXiv:2610.30951},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.30951},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2e66314e63fc; sha256 454c18f5eba43e82885eb8eca48bbfdd3cdafaa1f6c01ee9940f22b95959fc1f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.30951. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Sparse attention reads only a small subset of the key-value cache, and how to treat the unread tail remains a key challenge. Discarding the tail biases the output, and sampling it as previous approaches do fails at reads of 5% and below, where the tail likely holds heavy hitters the selector missed. Our key observation is that the simplest possible proxy, one token that folds the entire tail into its count-aware mean, already yields lower output error than a sample at every read behind a practical selector. We further show in closed form that at low reads sampling nearly doubles the weight error of discarding while a single proxy stays below it. Building on these findings, we explore approximating the dense attention distribution with a few proxies that target the tail. We propose OpenAttention, a pipeline of clustering, ErrorGate and top-k selection followed by a count-aware mean proxy. To implement clustering and selection both efficiently and effectively, we design training-free Query-Geometry Clustering (QGC) and a Query-Geometry Selector (QGS), which compress the keys while respecting the query directions. Additionally, unlike prior methods that rank clusters by centroid alone, ErrorGate allocates the exact read by the estimated weight error of each cluster. On RULER-32K HARD with Llama-3.1-8B, OpenAttention retains 99% of dense accuracy at a 1% read. It also achieves state-of-the-art accuracy across models, context lengths and datasets. Our customized attention kernel, this http URL , runs the decode attention stage 10$\times$ faster than dense attention at a 1% read. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.30951 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-23T15:09:08.573Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).