Metraipolis › archive › dc_024ecf0c1b75

TVP: TV is Almost All You Need for Speculative Decoding — arXiv:2611.00588

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.00588
License (as archived)
CC0-1.0
Archived
2026-11-03T11:11:41.978Z
SHA-256 of the archived text
df2ae56a956f1d9c2619812f38f5a5a62d7d2a6244aa4051ff2dadc061a047ca
Stable URL of this page
https://metraipolis.com/documento/dc_024ecf0c1b75

How to cite

TVP: TV is Almost All You Need for Speculative Decoding — arXiv:2611.00588 — arXiv (en), https://arxiv.org/abs/2611.00588 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_024ecf0c1b75 (Metraipolis archive dc_024ecf0c1b75, sha256 df2ae56a956f1d9c2619812f38f5a5a62d7d2a6244aa4051ff2dadc061a047ca)
BibTeX
@misc{dc024ecf0c1b75,
  title = {TVP: TV is Almost All You Need for Speculative Decoding — arXiv:2611.00588},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.00588},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_024ecf0c1b75; sha256 df2ae56a956f1d9c2619812f38f5a5a62d7d2a6244aa4051ff2dadc061a047ca},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.00588. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Speculative decoding accelerates LLM inference with a drafter whose proposed tokens are verified by the target model in parallel. Under lossless speculative sampling, a draft token is accepted with probability one minus the total variation (TV) distance between the target and draft distributions, yet drafters are often trained with cross-entropy (CE) or KL. We first show that under matched and well-tuned training, TV matches or surpasses several published draft-training objectives when fine-tuning, and a fixed TV and KL mixture does the same when training from scratch. We identify two remaining problems, TV's vanishing gradients on undercovered tokens and the prefix products that starve later positions in acceptance-length objectives. To address both, we derive TVP, a loss family based on expected acceptance length, whose power parameter interpolates between coverage and acceptance oriented objectives. Annealing the power from zero to one gives the highest or tied-highest acceptance length across three drafter architectures and three target models, including longer drafts and block verification. TVP improves acceptance length over CE by 9.8% when fine-tuning, and over the public DFlash2 training recipes by 6.2 to 12.9%.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2611.00588 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:59:52.384Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).