Metraipolis › archive › dc_5dc405fdfcd6

RouteSparse: Input-Conditional Pattern Routing for Budgeted Long-Context Prefilling — arXiv:2608.29058

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.29058
License (as archived): CC0-1.0
Archived: 2026-10-17T08:06:51.032Z
SHA-256 of the archived text: 9ca8fe8ea272eb2621673a6d4a6e69b9cbd811d73345250a91eb095520287387
Stable URL of this page: https://metraipolis.com/documento/dc_5dc405fdfcd6

How to cite

RouteSparse: Input-Conditional Pattern Routing for Budgeted Long-Context Prefilling — arXiv:2608.29058 — arXiv (en), https://arxiv.org/abs/2608.29058 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5dc405fdfcd6 (Metraipolis archive dc_5dc405fdfcd6, sha256 9ca8fe8ea272eb2621673a6d4a6e69b9cbd811d73345250a91eb095520287387)

BibTeX

@misc{dc5dc405fdfcd6,
  title = {RouteSparse: Input-Conditional Pattern Routing for Budgeted Long-Context Prefilling — arXiv:2608.29058},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.29058},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5dc405fdfcd6; sha256 9ca8fe8ea272eb2621673a6d4a6e69b9cbd811d73345250a91eb095520287387},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.29058. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Dynamic sparse attention can reduce the quadratic cost of long-context prefilling without changing model weights. MInference assigns each attention head one pattern offline and estimates that pattern's sparse indices for every prompt. This design is efficient, but it assumes that a head's preferred pattern and sparsity budget remain suitable across inputs. We introduce RouteSparse, which routes each head and prompt segment among a small library of GPU-efficient sparse patterns. A low-cost probe estimates pattern utility and uncertainty; a latency-aware router then selects a pattern and budget, while uncertain cases fall back to a denser mask. We formulate routing as constrained risk minimization, derive an attention-output error certificate from omitted probability mass, and evaluate the method on long-context retrieval, question answering, summarization, and language modeling. On Llama 3.1-8B-Instruct with 128K-token prompts, RouteSparse achieves $6.5\times$ dense prefill speed with a 0.2-point RULER drop relative to dense attention, compared with $7.3\times$ speed and a 1.6-point drop for fixed per-head routing. Ablations confirm that input-conditional routing, hardware profiling, and selective dense fallback each contribute to the quality--latency tradeoff.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.29058 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T07:47:52.670Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).