Metraipolis › archive › dc_e8dac82183ce

To Copy or Not to Copy: Controlling Speculative Decoding via Intrinsic Model Signals — arXiv:2609.20186

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.20186
License (as archived)
CC0-1.0
Archived
2026-10-10T03:48:09.925Z
SHA-256 of the archived text
1ee9e134af88879d4f53ba868d3a1f165908a2d3c26b10ab416469dde3685531
Stable URL of this page
https://metraipolis.com/documento/dc_e8dac82183ce

How to cite

To Copy or Not to Copy: Controlling Speculative Decoding via Intrinsic Model Signals — arXiv:2609.20186 — arXiv (en), https://arxiv.org/abs/2609.20186 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e8dac82183ce (Metraipolis archive dc_e8dac82183ce, sha256 1ee9e134af88879d4f53ba868d3a1f165908a2d3c26b10ab416469dde3685531)
BibTeX
@misc{dce8dac82183ce,
  title = {To Copy or Not to Copy: Controlling Speculative Decoding via Intrinsic Model Signals — arXiv:2609.20186},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.20186},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e8dac82183ce; sha256 1ee9e134af88879d4f53ba868d3a1f165908a2d3c26b10ab416469dde3685531},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.20186. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Speculative Decoding (SD) has significantly accelerated Large Language Model (LLM) inference, yet existing approaches face a fundamental tradeoff between two drafting strategies: neural drafting and context-based copying. Neural drafts (e.g., EAGLE3) provide robust performance across diverse text settings, while copy-based methods achieve higher speedups in copy-intensive regimes by generating candidates faster and exploiting long repetition spans for near-perfect speculation. We analyze existing copy-based methods and find that they are prone to accidental repetitions where surface-level n-gram overlap does not reflect a structural intent to copy, leading to false-positive triggers that ultimately degrade throughput. We introduce SwitchSD, an adaptive framework that treats copying as a latent control signal of the LLM. By training lightweight probes on the target model's internal representations, SwitchSD identifies genuine copy-intent with high precision (AUC > 0.99). This allows the system to dynamically switch between neural drafting (e.g., EAGLE) and context-based copying. Our results across Llama and Qwen families demonstrate throughput gains of up to 15% over state-of-the-art baselines like EAGLE3, effectively turning copying from a noisy heuristic into a principled, model-aware decoding regime.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-18 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.20186 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-10T03:24:08.152Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).