Metraipolis › archive › dc_dbaf87bf0b50

Nucleus Speculative Decoding: Plausibility-Aware Verification Beyond Exact Distribution — arXiv:2610.07822

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.07822
License (as archived): CC0-1.0
Archived: 2026-10-07T11:23:13.251Z
SHA-256 of the archived text: 81edca20e16fb87878efb4f4d9d19b4ca6c01545245a426b509440a2007ee7c9
Stable URL of this page: https://metraipolis.com/documento/dc_dbaf87bf0b50

How to cite

Nucleus Speculative Decoding: Plausibility-Aware Verification Beyond Exact Distribution — arXiv:2610.07822 — arXiv (en), https://arxiv.org/abs/2610.07822 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dbaf87bf0b50 (Metraipolis archive dc_dbaf87bf0b50, sha256 81edca20e16fb87878efb4f4d9d19b4ca6c01545245a426b509440a2007ee7c9)

BibTeX

@misc{dcdbaf87bf0b50,
  title = {Nucleus Speculative Decoding: Plausibility-Aware Verification Beyond Exact Distribution — arXiv:2610.07822},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.07822},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dbaf87bf0b50; sha256 81edca20e16fb87878efb4f4d9d19b4ca6c01545245a426b509440a2007ee7c9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.07822. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Speculative decoding accelerates autoregressive generation by using a lightweight draft model to propose multiple tokens that are verified by a target model in parallel. However, the standard acceptance rule focuses on exact distribution correction and rejects tokens that remain highly plausible under the target model when the draft model assigns excess probability. This conservative verification limits the number of draft tokens retained after each verification forward pass. We introduce Nucleus Speculative Decoding (NSD), a relaxed verification method that incorporates target-model plausibility into speculative decoding. NSD accepts a draft token if it satisfies the standard acceptance rule or belongs to the target model's nucleus. We theoretically characterize the distributional deviation introduced by our method and show that the single-step error is exactly determined by the draft model's excess probability within the target nucleus. We further derive sequence-level fidelity bounds that quantify how local deviations accumulate over autoregressive decoding. Experiments across multiple target models and proposal mechanisms demonstrate that NSD consistently improves speculative decoding efficiency while maintaining competitive task performance. Our method achieves throughput speedups of up to $5.16\times$ over autoregressive decoding and up to $3.15\times$ over standard speculative decoding. These improvements coincide with longer accepted lengths, allowing more output tokens to share the cost of each target verification pass. Analysis shows that plausibility-aware verification provides an effective approach for relaxed verification and speculative decoding efficiency. Our code is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.07822 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T11:15:39.434Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).