Metraipolis › archive › dc_079f3da49758
SpecOPD: Branch-Parallel On-Policy Distillation for Speculative Decoding — arXiv:2611.02846
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.02846
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T10:24:02.163Z
- SHA-256 of the archived text
b02e850b88debdf9fcb601ccf34ba7c566884bd8a545813d8bb360c2c0bbd4c9- Stable URL of this page
https://metraipolis.com/documento/dc_079f3da49758
How to cite
SpecOPD: Branch-Parallel On-Policy Distillation for Speculative Decoding — arXiv:2611.02846 — arXiv (en), https://arxiv.org/abs/2611.02846 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_079f3da49758 (Metraipolis archive dc_079f3da49758, sha256 b02e850b88debdf9fcb601ccf34ba7c566884bd8a545813d8bb360c2c0bbd4c9)
BibTeX
@misc{dc079f3da49758,
title = {SpecOPD: Branch-Parallel On-Policy Distillation for Speculative Decoding — arXiv:2611.02846},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.02846},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_079f3da49758; sha256 b02e850b88debdf9fcb601ccf34ba7c566884bd8a545813d8bb360c2c0bbd4c9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02846. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Speculative decoding (SD) accelerates large language model inference by reducing the sequential cost of token-by-token generation. Advanced SD methods reduce proposal cost with parallel drafters that predict entire blocks in one forward pass, but overall speedup still depends on acceptance length. On-policy distillation (OPD) can push well-trained drafters beyond acceptance-length plateaus, yet conventional pipelines collect supervision through sequential draft-and-verify rollouts. Our key insight is that on-policy supervision need not inherit these dependencies: a draft block can be supervised using only its reference prefix and draft-sampled history. We propose SpecOPD, a branch-parallel OPD framework that parallelizes draft sampling and target supervision across independent branches drawn from the evolving drafter policy at reference anchors. We pack these branches into proposal trees to share target-prefix computation, using tree masks for full attention, branch-consistent indexer groups for sparse attention, and indexed state replay for linear attention to preserve each branch's history. We train the drafter with an E2E-TV surrogate that directly targets acceptance length. Our study spans 8B full-attention (GQA), 27B full-linear (GQA--GDN), and 125B sparse-linear (QSA--GDN) targets, three parallel draft architectures (DFlash, DSpark, and DFlash V2), and four task domains (dialogue, coding, mathematics, and agentic). SpecOPD achieves up to 41.45% higher mean acceptance length, $16.09\times$ supervised-token throughput over sequential-rollout OPD under native configurations, and up to $3.56\times$ generation speedup over autoregressive decoding. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2611.02846 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T10:19:49.915Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).