Metraipolis › archive › dc_bc46cf0e5bf0

Adaptive Multi-Token Prediction: How far should we go? — arXiv:2610.28619

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.28619
License (as archived)
CC0-1.0
Archived
2026-10-21T07:07:13.150Z
SHA-256 of the archived text
dbe271d27db9846e090b52495dffba37e232914eb873662cb00473a45ec12b29
Stable URL of this page
https://metraipolis.com/documento/dc_bc46cf0e5bf0

How to cite

Adaptive Multi-Token Prediction: How far should we go? — arXiv:2610.28619 — arXiv (en), https://arxiv.org/abs/2610.28619 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bc46cf0e5bf0 (Metraipolis archive dc_bc46cf0e5bf0, sha256 dbe271d27db9846e090b52495dffba37e232914eb873662cb00473a45ec12b29)
BibTeX
@misc{dcbc46cf0e5bf0,
  title = {Adaptive Multi-Token Prediction: How far should we go? — arXiv:2610.28619},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.28619},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bc46cf0e5bf0; sha256 dbe271d27db9846e090b52495dffba37e232914eb873662cb00473a45ec12b29},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.28619. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multi-token prediction (MTP) can accelerate language-model decoding by drafting several tokens and verifying them in a single target-model pass. However, the best speculation horizon changes with context: longer drafts offer greater amortization when reliable, but can waste compute on suffixes discarded after a rejection. Unlike autoregressive drafting, where a stopping rule can decide token by token, MTP must fix the horizon before drafting, as all heads fire in the same forward pass. We therefore formulate horizon selection as an online control problem, using verification feedback to learn how far the draft can be trusted in each context, and introduce Adaptive Multi-Token Prediction (AMTP): a lightweight controller that requires no retraining and leaves the exact verifier unchanged. To guide horizon selection, we characterize the expected number of tokens committed at each horizon via acceptance survival, the probability that a draft prefix is fully accepted. Before each round, AMTP predicts how far the draft is likely to survive and picks the horizon with the highest expected tokens per second, learning freely from every verification pass. Theoretically, we show that any online learner's prediction error translates into throughput regret against the best horizon for each context, and in turn into a bound on wall-clock completion time. With online gradient descent, this regret is sublinear. Across five draft-target pairings built on Llama-3.1-8B-Instruct with Medusa heads, Meta's MTP models, and Qwen3-4B-Inst-2507-MTP, evaluated on AIME 2025 and GSM8K, AMTP decodes up to 4.1x faster than standard decoding and matches or outperforms the fixed window in 8 of 10 settings, with throughput gains of up to 10%.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG); Systems and Control (eess.SY).
Pagina del lavoro: https://arxiv.org/abs/2610.28619 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-21T07:05:43.497Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).