Metraipolis › archive › dc_e8dac82183ce
To Copy or Not to Copy: Controlling Speculative Decoding via Intrinsic Model Signals — arXiv:2609.20186
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.20186
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-10T03:48:09.925Z
- SHA-256 of the archived text
1ee9e134af88879d4f53ba868d3a1f165908a2d3c26b10ab416469dde3685531- Stable URL of this page
https://metraipolis.com/documento/dc_e8dac82183ce
How to cite
To Copy or Not to Copy: Controlling Speculative Decoding via Intrinsic Model Signals — arXiv:2609.20186 — arXiv (en), https://arxiv.org/abs/2609.20186 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e8dac82183ce (Metraipolis archive dc_e8dac82183ce, sha256 1ee9e134af88879d4f53ba868d3a1f165908a2d3c26b10ab416469dde3685531)
BibTeX
@misc{dce8dac82183ce,
title = {To Copy or Not to Copy: Controlling Speculative Decoding via Intrinsic Model Signals — arXiv:2609.20186},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.20186},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e8dac82183ce; sha256 1ee9e134af88879d4f53ba868d3a1f165908a2d3c26b10ab416469dde3685531},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.20186. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Speculative Decoding (SD) has significantly accelerated Large Language Model (LLM) inference, yet existing approaches face a fundamental tradeoff between two drafting strategies: neural drafting and context-based copying. Neural drafts (e.g., EAGLE3) provide robust performance across diverse text settings, while copy-based methods achieve higher speedups in copy-intensive regimes by generating candidates faster and exploiting long repetition spans for near-perfect speculation. We analyze existing copy-based methods and find that they are prone to accidental repetitions where surface-level n-gram overlap does not reflect a structural intent to copy, leading to false-positive triggers that ultimately degrade throughput. We introduce SwitchSD, an adaptive framework that treats copying as a latent control signal of the LLM. By training lightweight probes on the target model's internal representations, SwitchSD identifies genuine copy-intent with high precision (AUC > 0.99). This allows the system to dynamically switch between neural drafting (e.g., EAGLE) and context-based copying. Our results across Llama and Qwen families demonstrate throughput gains of up to 15% over state-of-the-art baselines like EAGLE3, effectively turning copying from a noisy heuristic into a principled, model-aware decoding regime. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-18 in cs.CL). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.20186 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-10T03:24:08.152Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).