Metraipolis › archive › dc_8cb8bf3f9877
Evaluating Losslessness in Speculative Decoding Under Finite-Precision Inference — arXiv:2609.15504
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.15504
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T22:53:50.449Z
- SHA-256 of the archived text
62233c17a02f43de81a0e61af951a855531b25942d845b1a47b5edf0940d2a8f- Stable URL of this page
https://metraipolis.com/documento/dc_8cb8bf3f9877
How to cite
Evaluating Losslessness in Speculative Decoding Under Finite-Precision Inference — arXiv:2609.15504 — arXiv (en), https://arxiv.org/abs/2609.15504 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8cb8bf3f9877 (Metraipolis archive dc_8cb8bf3f9877, sha256 62233c17a02f43de81a0e61af951a855531b25942d845b1a47b5edf0940d2a8f)
BibTeX
@misc{dc8cb8bf3f9877,
title = {Evaluating Losslessness in Speculative Decoding Under Finite-Precision Inference — arXiv:2609.15504},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.15504},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8cb8bf3f9877; sha256 62233c17a02f43de81a0e61af951a855531b25942d845b1a47b5edf0940d2a8f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.15504. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Lossless speculative decoding is typically defined at the algorithmic level: a speculative procedure proposes multiple tokens and a verification procedure is designed to preserve the output trajectory of an autoregressive reference model exactly. In practical neural inference, however, this guarantee is implemented using finite-precision floating-point computations, and discrete token selection can amplify small numerical differences into divergent generation trajectories. We investigate this distinction using Orthrus, a hybrid autoregressive-diffusion architecture that performs self-drafting and self-verification within a frozen autoregressive backbone, as a representative case study. Across 1,190 prompts from 12 domains, exact trajectory matching under BF16 occurs for only 45\% of the authors' checkpoint generations and 43% of those from our independently trained model. The probability of matching is strongly associated with the response-conditional perplexity of the autoregressive reference, indicating that trajectory divergence is not uniform across inputs. Despite these divergences, Orthrus does not exhibit systematic degradation on the evaluated downstream tasks. In contrast, FP32 inference yields exact trajectory matching on all evaluated prompts. These results demonstrate a gap between algorithmic losslessness and its implementation under finite-precision arithmetic, and motivate evaluating lossless speculative decoding at the level of exact generation trajectories as well as downstream task performance. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-15 in cs.AI). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.15504 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T22:29:03.861Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-15?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).