Metraipolis › archive › dc_444ecaa78f06
JetSpec++: Foresight Parallel Drafting with Self Refinements — arXiv:2610.19792
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.19792
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T20:34:41.799Z
- SHA-256 of the archived text
c83a0995827687745be6a0a0a3a1bfb80aaf6db123b1e517755bb8a6e3131af4- Stable URL of this page
https://metraipolis.com/documento/dc_444ecaa78f06
How to cite
JetSpec++: Foresight Parallel Drafting with Self Refinements — arXiv:2610.19792 — arXiv (en), https://arxiv.org/abs/2610.19792 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_444ecaa78f06 (Metraipolis archive dc_444ecaa78f06, sha256 c83a0995827687745be6a0a0a3a1bfb80aaf6db123b1e517755bb8a6e3131af4)
BibTeX
@misc{dc444ecaa78f06,
title = {JetSpec++: Foresight Parallel Drafting with Self Refinements — arXiv:2610.19792},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.19792},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_444ecaa78f06; sha256 c83a0995827687745be6a0a0a3a1bfb80aaf6db123b1e517755bb8a6e3131af4},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.19792. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Speculative decoding's effectiveness is governed by two quantities: the acceptance rate of drafted tokens and the cost of producing them. We show the two objectives can be co-optimized with algorithm-system co-design, and present JetSpec++, which retains a strong learned draft model while hiding its cost almost entirely. JetSpec++ combines three components: intra-block self refinement, which trains the draft model to iteratively correct its own parallel predictions within a block of tokens; inter-block foresight drafting, which begins the next block before the current one is verified; and pipelined drafting and verification, which overlaps the two stages so drafting's marginal wall-clock cost approaches zero. In comparison with state-of-the-art speculative decoding methods, JetSpec++ attains a wall-clock speedup of up to $7.45\times$ over autoregressive (AR) decoding and $1.54\times$ over DSpark, with up to 1,361 tokens per second for a single request on GB200, and an average acceptance length of 8.94 across seven benchmarks, while preserving the target model's output distribution. Our code and models are available at this https URL . Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.19792 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T19:36:18.224Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).