Metraipolis › archive › dc_f0cb3d979aaa

ASPIRE: Asynchronous Batched Self-Speculative Decoding for Long-Context LLM Inference — arXiv:2609.17943

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.17943
License (as archived)
CC0-1.0
Archived
2026-10-15T13:49:50.828Z
SHA-256 of the archived text
0231c73a41da43e2e62e433f74f3bb9992fbc0eb2418c118e98df7f32d7aad9a
Stable URL of this page
https://metraipolis.com/documento/dc_f0cb3d979aaa

How to cite

ASPIRE: Asynchronous Batched Self-Speculative Decoding for Long-Context LLM Inference — arXiv:2609.17943 — arXiv (en), https://arxiv.org/abs/2609.17943 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f0cb3d979aaa (Metraipolis archive dc_f0cb3d979aaa, sha256 0231c73a41da43e2e62e433f74f3bb9992fbc0eb2418c118e98df7f32d7aad9a)
BibTeX
@misc{dcf0cb3d979aaa,
  title = {ASPIRE: Asynchronous Batched Self-Speculative Decoding for Long-Context LLM Inference — arXiv:2609.17943},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.17943},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f0cb3d979aaa; sha256 0231c73a41da43e2e62e433f74f3bb9992fbc0eb2418c118e98df7f32d7aad9a},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.17943. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Long-context LLM inference is bottlenecked by attention, whose repeated KV-cache reads make decoding memory-bound. Self-speculative decoding alleviates this by drafting tokens with sparse attention and verifying them with full attention, but existing batched methods remain synchronized: all requests in a batch share a single draft-verify schedule, even though the optimal draft length varies widely across requests and changes dynamically within each request. We propose ASPIRE, a non-synchronized batched self-speculative decoding framework built on three components. First, a unified mixed forward allows drafting and verifying requests to coexist in the same batched forward pass, removing the need for global draft-verify phases. Second, a lightweight online speculation scheduler uses per-request acceptance-rate estimates and a batch-aware cost model to let each request independently choose when to verify. Third, an intra-draft refresh layer performs full attention at a single designated layer during drafting, updating the sparse context at every draft step to reduce staleness during drafting. Across three models and five reasoning and long-context benchmarks, ASPIRE achieves $1.70$-$4.58\times$ speedup in decoding throughput over autoregressive baselines and improves average speedup by approximately $27\%$ over the strongest prior self-speculative baselines.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-17 in cs.CL).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL); Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2609.17943 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T13:48:24.399Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).