Metraipolis › archive › dc_e34fc0bad259

When Should Diffusion Language Models Reconsider Their Commitments? — arXiv:2611.11551

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.11551
License (as archived): CC0-1.0
Archived: 2026-11-10T18:43:31.252Z
SHA-256 of the archived text: 7a60bf8963f4a3b8152401daa48c416d996b4a9b9b3c2352fc99cc1c7558fd23
Stable URL of this page: https://metraipolis.com/documento/dc_e34fc0bad259

How to cite

When Should Diffusion Language Models Reconsider Their Commitments? — arXiv:2611.11551 — arXiv (en), https://arxiv.org/abs/2611.11551 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e34fc0bad259 (Metraipolis archive dc_e34fc0bad259, sha256 7a60bf8963f4a3b8152401daa48c416d996b4a9b9b3c2352fc99cc1c7558fd23)

BibTeX

@misc{dce34fc0bad259,
  title = {When Should Diffusion Language Models Reconsider Their Commitments? — arXiv:2611.11551},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.11551},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e34fc0bad259; sha256 7a60bf8963f4a3b8152401daa48c416d996b4a9b9b3c2352fc99cc1c7558fd23},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.11551. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Masked Diffusion Models (MDMs) are well known for fast text generation via parallel decoding. However, independent parallel sampling can commit to inconsistent predictions, and less is known about how to selectively revise these early token commitments. In this work, we introduce $\underline{\mathrm{P}}$lanned $\underline{\mathrm{U}}$nmasking-and-$\underline{\mathrm{R}}$evision Joint $\underline{\mathrm{L}}$earning (PURL) for MDMs, a training-based framework that seamlessly unifies parallel generation with selective token revision. We derive a normalized joint transition whose likelihood factorizes into edit selection, unmask selection, and token prediction. These components are trained jointly on partially masked sequences constructed according to the model's own unmasking policy. During inference, all three decisions are supported by a single shared forward pass, enabling the model to decode masked tokens and revise noisy tokens. Across logic puzzles, mathematical reasoning, and code generation, PURL improves accuracy at matched unmask budgets, with relative gains up to 47.4% accuracy improvement and up to an $8\times$ reduction in scheduled decoding rounds. Beyond these gains, our analysis further reveals general principles of revisable decoding: token edits become more effective as context accumulates; early edits can steer unfinished trajectories, whereas later edits tend to remain localized; and effective edits therefore depend on both calibrating when to intervene and learning where and how to revise. Together, these suggest that effective parallel generation depends not on revising more tokens, but on making the right commitments revisable at the right time.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.11551 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-10T18:37:37.648Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).