Metraipolis › archive › dc_e4e73db41981

What Makes a Good Unmasking Policy? A Controlled Study of Learned Unmasking in Diffusion Language Models — arXiv:2610.31576

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.31576
License (as archived): CC0-1.0
Archived: 2026-10-25T20:54:33.217Z
SHA-256 of the archived text: 61f66ccb5cd900388897554665c45e07149e8f453482da32ea7cfb4e7c6dd9b8
Stable URL of this page: https://metraipolis.com/documento/dc_e4e73db41981

How to cite

What Makes a Good Unmasking Policy? A Controlled Study of Learned Unmasking in Diffusion Language Models — arXiv:2610.31576 — arXiv (en), https://arxiv.org/abs/2610.31576 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e4e73db41981 (Metraipolis archive dc_e4e73db41981, sha256 61f66ccb5cd900388897554665c45e07149e8f453482da32ea7cfb4e7c6dd9b8)

BibTeX

@misc{dce4e73db41981,
  title = {What Makes a Good Unmasking Policy? A Controlled Study of Learned Unmasking in Diffusion Language Models — arXiv:2610.31576},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.31576},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e4e73db41981; sha256 61f66ccb5cd900388897554665c45e07149e8f453482da32ea7cfb4e7c6dd9b8},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.31576. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Masked diffusion language models generate tokens in parallel, but their efficiency depends on which predictions are committed. Learned unmasking policies improve the speed--accuracy trade-off, yet differ in supervision, inputs, and capacity, obscuring which choices matter. We view supervised policies as distilling a decoding procedure through a teacher and examine independently what the teacher labels as committable, on which states the policy imitates it, what the policy observes, and how much the number of function evaluations (NFE) must decrease to offset policy overhead. We find that labeling predictions matching the final output is overly permissive at low NFE, and hidden states are weak alone but can complement confidence. About a 10% NFE reduction suffices to offset policy overhead for policies up to 7.5M parameters across tested settings. We therefore train a policy on trajectory-contraction targets, which accept only commitments staying on a reference trajectory, using confidence and hidden states. On LLaDA-8B-Instruct, the resulting 856K-parameter policy achieves the best accuracy--NFE frontier and highest throughput on GSM8K, is competitive at moderate targets on MATH-500 and HumanEval, and largely transfers to Dream-7B-Instruct.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.31576 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-25T20:47:56.944Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).