Metraipolis › archive › dc_9bf210dd4663
Simple Diffusion Language Models Are More Effective Few-Step Generators Than Reported — arXiv:2609.33947
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.33947
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T21:53:14.428Z
- SHA-256 of the archived text
a01e25e75c93b1663d0ac049d4d7d23bd187c393b517b8419333c6ce268ead01- Stable URL of this page
https://metraipolis.com/documento/dc_9bf210dd4663
How to cite
Simple Diffusion Language Models Are More Effective Few-Step Generators Than Reported — arXiv:2609.33947 — arXiv (en), https://arxiv.org/abs/2609.33947 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_9bf210dd4663 (Metraipolis archive dc_9bf210dd4663, sha256 a01e25e75c93b1663d0ac049d4d7d23bd187c393b517b8419333c6ce268ead01)
BibTeX
@misc{dc9bf210dd4663,
title = {Simple Diffusion Language Models Are More Effective Few-Step Generators Than Reported — arXiv:2609.33947},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.33947},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_9bf210dd4663; sha256 a01e25e75c93b1663d0ac049d4d7d23bd187c393b517b8419333c6ce268ead01},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33947. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Diffusion language models (DLMs) promise fast parallel generation, yet high-quality samples often require large number of refinement steps, which diminishes their advantage in practice. This has led to massive interest in and rapid development of new methods for effective few-step generation. We show that much of the supposed quality gap at few steps can instead arise from a suboptimally configured sampler. Modest sampler sharpening, without any model retraining, enables a couple years old masked DLM to rival supposedly far improved successors. This differently sampled DLM in fact achieves lower generative perplexity in just 16 steps than what its standard sampler obtains with 1024, while improving both judged quality and semantic diversity. We further show that conventional per-output metrics can fundamentally obscure these gains, since any optimal trade-off between two such metrics can be attained by a generator supported on at most two outputs. We subsequently introduce GroupEval, which separately evaluates quality and across-output semantic diversity, and offers fresh insights including uncovering how 1.5-4.7x perplexity gains of a distilled model yield no corresponding quality gain. Finally, we explain why sharpening helps: parallel unmasking destroys dependencies among simultaneously generated tokens, creating a gap between prediction and generation. We prove that pervasive temperature choice of one is generically suboptimal under parallel sampling even for an exact denoiser, and that worse predictions can yield better samples. Through these results, we argue for a broader evaluation principle of treating the deployed generator as the object of comparison, benchmarking it against tuned baselines, and assessing quality and diversity jointly and with more human-aligned measures. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.33947 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T21:27:08.569Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).