Metraipolis › archive › dc_89a557cb1cdb

Multi-layer Indexer: Overlapping Selection and Attention in DeepSeek Sparse Attention — arXiv:2611.03768

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.03768
License (as archived)
CC0-1.0
Archived
2026-11-04T21:39:37.616Z
SHA-256 of the archived text
0ff594313a8d8dfa45c0868c2872803bc8700414a88ebcd11e444f2d5e575a53
Stable URL of this page
https://metraipolis.com/documento/dc_89a557cb1cdb

How to cite

Multi-layer Indexer: Overlapping Selection and Attention in DeepSeek Sparse Attention — arXiv:2611.03768 — arXiv (en), https://arxiv.org/abs/2611.03768 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_89a557cb1cdb (Metraipolis archive dc_89a557cb1cdb, sha256 0ff594313a8d8dfa45c0868c2872803bc8700414a88ebcd11e444f2d5e575a53)
BibTeX
@misc{dc89a557cb1cdb,
  title = {Multi-layer Indexer: Overlapping Selection and Attention in DeepSeek Sparse Attention — arXiv:2611.03768},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.03768},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_89a557cb1cdb; sha256 0ff594313a8d8dfa45c0868c2872803bc8700414a88ebcd11e444f2d5e575a53},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.03768. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Sparse attention cuts the cost of long-context inference by making the attention matrix sparse: each query attends only a subset of the sequence, chosen by a fixed pattern or, in recent production LLMs, learned per query. In DeepSeek Sparse Attention (DSA), the attention of DeepSeek-V3.2 and the GLM-5 family, every layer is preceded by a lightning indexer that scores the whole cache and keeps the k=2048 highest-scoring tokens for the layer to attend. At long context this indexer scan, not the sparse attention it feeds, is where the decode time goes. But adjacent layers select nearly the same tokens, so a layer can attend the selection of the previous indexer layer instead of its own. This removes the dependency: each indexer then runs concurrently with the attention of its layer. The resulting Multi-layer Indexer (MLI) needs no retraining and no new weights. Its default realization changes no kernel, running inside SGLang's production decode graph; an optional fused decode kernel accelerates it further on DeepSeek-V3.2. MLI composes with IndexCache (the cross-layer reuse GLM-5.2 ships as IndexShare): IndexCache drops the indexer from some layers, and MLI hides the scan on the rest. We serve DeepSeek-V3.2 and GLM-5 under IndexCache's protocol at retentions 1/2 to 1/32 (the fraction of layers that keep an indexer) and contexts 10K to 195K. At equal retention the composition decodes faster than IndexCache: up to 14.5% faster on DeepSeek-V3.2 and 9.8% on GLM-5 at 1/2, and 7.9% and 5.1% at 1/4, largest where IndexCache keeps the most indexers and growing with context. At retention 1/r it is faster still than IndexCache at 1/(2r) while keeping twice the fresh selections: at 128K it reaches 1.32x the unmodified model's decode throughput at 1/4, above IndexCache's 1.28x at 1/8. Its LongBench, RULER and LongBench v2 accuracy stays within a few percentage points of IndexCache's.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.03768 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T21:35:49.617Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).