Metraipolis › archive › dc_7ff4031a255d
ValuSpec: Plug-and-Play Candidate Valuation before Target Verification for Tree-Based Speculative Decoding — arXiv:2610.31338
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.31338
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-23T15:13:22.562Z
- SHA-256 of the archived text
5515515275bbea2d9bff56fe5ff1d7e38be17b1cafc2977d0b7f09d52aa20ea3- Stable URL of this page
https://metraipolis.com/documento/dc_7ff4031a255d
How to cite
ValuSpec: Plug-and-Play Candidate Valuation before Target Verification for Tree-Based Speculative Decoding — arXiv:2610.31338 — arXiv (en), https://arxiv.org/abs/2610.31338 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7ff4031a255d (Metraipolis archive dc_7ff4031a255d, sha256 5515515275bbea2d9bff56fe5ff1d7e38be17b1cafc2977d0b7f09d52aa20ea3)
BibTeX
@misc{dc7ff4031a255d,
title = {ValuSpec: Plug-and-Play Candidate Valuation before Target Verification for Tree-Based Speculative Decoding — arXiv:2610.31338},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.31338},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7ff4031a255d; sha256 5515515275bbea2d9bff56fe5ff1d7e38be17b1cafc2977d0b7f09d52aa20ea3},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.31338. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Tree-based speculative decoding accelerates large language model inference by verifying multiple candidate branches in parallel. However, in practical serving scenarios, similar requests repeatedly induce overlapping candidate branches. While retrieval-based methods mitigate this by reusing historical drafts, the resulting candidate trees often contain many low-quality branches as the candidate tree grows, which reduces verification efficiency and wastes target-side computation. To address this, we propose ValuSpec, a plug-and-play candidate valuation mechanism that filters low-quality branches from a supplied candidate tree before expensive target-side verification. ValuSpec introduces transfer-based supervision to align filtering with the target model's behavior. A filtering threshold is estimated from the target model's greedy trajectories on a calibration split, without any parameter training. The target model then verifies the filtered tree under the output-identical greedy rule, so accepted tokens are guaranteed to match standard autoregressive decoding exactly. Experimental results on HumanEval, GSM8K, Dolly, and SpecBench show that, in our main evaluation, ValuSpec achieves up to 2.38x end-to-end speedup in retrieval-based tree settings. Furthermore, integrating ValuSpec into the EAGLE-3 dynamic tree framework yields up to 2.78x end-to-end speedup without modifying its tree construction strategy, demonstrating that ValuSpec serves as a generally applicable and plug-and-play candidate valuation module. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.31338 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-23T15:09:08.455Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).