Metraipolis › archive › dc_5854dcf25cfc

Speculative Correction: Draft-then-Refine Decoding for Diffusion Language Models — arXiv:2608.02625

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.02625
License (as archived)
CC0-1.0
Archived
2026-10-31T12:17:26.205Z
SHA-256 of the archived text
6a0296e01b3f073e279ddc5d34e0c630d6bdce689518b5f9bef8808a38daf04e
Stable URL of this page
https://metraipolis.com/documento/dc_5854dcf25cfc

How to cite

Speculative Correction: Draft-then-Refine Decoding for Diffusion Language Models — arXiv:2608.02625 — arXiv (en), https://arxiv.org/abs/2608.02625 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5854dcf25cfc (Metraipolis archive dc_5854dcf25cfc, sha256 6a0296e01b3f073e279ddc5d34e0c630d6bdce689518b5f9bef8808a38daf04e)
BibTeX
@misc{dc5854dcf25cfc,
  title = {Speculative Correction: Draft-then-Refine Decoding for Diffusion Language Models — arXiv:2608.02625},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.02625},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5854dcf25cfc; sha256 6a0296e01b3f073e279ddc5d34e0c630d6bdce689518b5f9bef8808a38daf04e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.02625. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Diffusion language models (DLMs) can revise tokens bidirectionally, but standard decoding procedures often adapt them to left-to-right generation by producing text block by block. We study a simple plug-and-play inference pattern: first generate a complete draft, then refine the full response using bidirectional diffusion. Using LLaDA2.1-Flash and LLaDA2.1-Mini, we evaluate two configurations. In Flash-Flash, the same Flash model serves as both drafter and refiner, testing whether an existing model can improve its own block-autoregressive output through global refinement. In Mini-Flash, inspired by speculative decoding, we introduce speculative correction: Mini drafts a full response, and Flash revises it as an editable initialization. Flash-Flash improves GSM8K-384 accuracy from 0.848 to 0.899 while running 1.20 times faster than the selected Flash block-autoregressive baseline, and improves MBPP-384 from 0.545 to 0.693. Latency-window-matched Flash-only controls indicate that these gains persist after targeted tuning of block-autoregressive decoding. Causal ablations indicate that completed drafts provide useful initializations: refinement from a fully masked span performs poorly, full global refinement provides a clear additional gain on GSM8K, and local refinement captures much of the gain on MBPP and MATH. Mini-Flash provides useful quality-latency trade-offs, including MATH-384 performance of 0.294 versus 0.300 for Flash while running 2.17 times faster. These results support a Pareto-frontier interpretation rather than the claim that the heterogeneous cascade uniformly matches Flash quality. Overall, same-model draft-and-refine provides evidence that bidirectional refinement is a useful decoding primitive for DLMs, while speculative correction demonstrates a training-free route to fast DLM generation.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-05 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.02625 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T12:05:52.894Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-05?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).