Metraipolis › archive › dc_7ca468b2da0d

SparseEngine: Sparse-First Inference Engine — arXiv:2609.39068

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.39068
License (as archived): CC0-1.0
Archived: 2026-10-07T13:45:46.947Z
SHA-256 of the archived text: 06c64f4a445de5db257fa860490f5001026600f16a34f2ebefe095e8a0a688c1
Stable URL of this page: https://metraipolis.com/documento/dc_7ca468b2da0d

How to cite

SparseEngine: Sparse-First Inference Engine — arXiv:2609.39068 — arXiv (en), https://arxiv.org/abs/2609.39068 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7ca468b2da0d (Metraipolis archive dc_7ca468b2da0d, sha256 06c64f4a445de5db257fa860490f5001026600f16a34f2ebefe095e8a0a688c1)

BibTeX

@misc{dc7ca468b2da0d,
  title = {SparseEngine: Sparse-First Inference Engine — arXiv:2609.39068},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.39068},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7ca468b2da0d; sha256 06c64f4a445de5db257fa860490f5001026600f16a34f2ebefe095e8a0a688c1},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.39068. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Long-context LLM agents accumulate interaction histories that strain KV-cache memory and attention computation. Although sparse attention reduces these costs, heterogeneous cache representations and workflows hinder integration with existing inference engines, while prior sparse-serving abstractions support only specific layouts or workflows. We present SparseEngine, a ground-up, sparse-first inference engine whose shared lifecycle contract lets each method control its KV representation and computation while coordinating state transitions with common serving infrastructure. SparseEngine supports 15 methods across four categories and enables cross-request state management through Chain Cache, which resumes KV-eviction methods from retained history, and controllable Prefix-Cache Pruning, which removes KV from selected history regions while preserving logical-prefix matching. While maintaining method quality, SparseEngine delivers over 10x higher throughput with KV eviction, over 2.5x faster decoding at matched concurrency than vLLM, and over 2x end-to-end speedup on agent benchmarks. The code is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.39068 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T13:18:10.312Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).