Metraipolis › archive › dc_29bedf54834a

LeanSparse: Memory-Efficient Serving for Native Sparse-Attention LLMs — arXiv:2611.04671

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.04671
License (as archived)
CC0-1.0
Archived
2026-11-04T21:45:42.757Z
SHA-256 of the archived text
48f59896943b9fe43f50656d4c985b30404aa59d8a6b91a3b720e04811515252
Stable URL of this page
https://metraipolis.com/documento/dc_29bedf54834a

How to cite

LeanSparse: Memory-Efficient Serving for Native Sparse-Attention LLMs — arXiv:2611.04671 — arXiv (en), https://arxiv.org/abs/2611.04671 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_29bedf54834a (Metraipolis archive dc_29bedf54834a, sha256 48f59896943b9fe43f50656d4c985b30404aa59d8a6b91a3b720e04811515252)
BibTeX
@misc{dc29bedf54834a,
  title = {LeanSparse: Memory-Efficient Serving for Native Sparse-Attention LLMs — arXiv:2611.04671},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.04671},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_29bedf54834a; sha256 48f59896943b9fe43f50656d4c985b30404aa59d8a6b91a3b720e04811515252},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.04671. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Sparse attention limits the history read by each query, but serving long contexts still requires substantial GPU memory. Existing KV offloading systems retain the full history of indexer keys on GPU, limiting decode concurrency. Prefill's temporary memory demand can further delay admission even when enough memory remains for the same request's subsequent decode. We present LeanSparse, a serving system that reduces both decode's persistent memory use and prefill's temporary memory demand. Our design exploits two observations: prefill consumes historical data one layer at a time, and decode can identify promising candidates before comparing their original indexer scores. During prefill, LeanSparse streams layer histories through two reusable GPU buffers, overlapping next-layer loading with ongoing model computation. During decode, coarse-to-fine indexing searches compact representations and retrieves original indexer keys only for candidate refinement. Separate indexer key and KV caches reuse data across generation steps. We implement LeanSparse in SGLang and evaluate it on DeepSeek-V4-Flash. At 950K context length, LeanSparse supports 2.95$\times$ as many concurrent requests and achieves 2.14$\times$ decode throughput over HiSparse. On multi-turn Open-SWE traces, it reduces median time to first token from 21.6 to 0.6s and median session completion time by 17.7% relative to ECHO.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2611.04671 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T21:36:19.381Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).