Metraipolis › archive › dc_e71ea60d4c80
Representation-Space MMD for Diffusion Language Models — arXiv:2610.06648
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.06648
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T05:28:37.428Z
- SHA-256 of the archived text
b059b556bf18c2130f3aa5c595e7a0831cc8efd1b308636d8be6f40673d8ee37- Stable URL of this page
https://metraipolis.com/documento/dc_e71ea60d4c80
How to cite
Representation-Space MMD for Diffusion Language Models — arXiv:2610.06648 — arXiv, https://arxiv.org/abs/2610.06648 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e71ea60d4c80 (Metraipolis archive dc_e71ea60d4c80, sha256 b059b556bf18c2130f3aa5c595e7a0831cc8efd1b308636d8be6f40673d8ee37)
BibTeX
@misc{dce71ea60d4c80,
title = {Representation-Space MMD for Diffusion Language Models — arXiv:2610.06648},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.06648},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e71ea60d4c80; sha256 b059b556bf18c2130f3aa5c595e7a0831cc8efd1b308636d8be6f40673d8ee37},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.06648. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
We introduce a post-training method for diffusion language models (DLMs) that minimizes Maximum Mean Discrepancy (MMD) between generated and reference distributions in the feature space of a frozen pretrained DLM. To estimate MMD, we retain contextual features at individual token positions, obtaining multiple observations per sequence from a single extractor pass. We optimize this objective using policy gradients for discrete models and direct differentiation through generated latents for continuous models. In both cases, computing the loss directly from these features enables efficient post-training without full sampling trajectories or jointly trained auxiliary models. Experiments show lower generative perplexity at comparable entropy on OpenWebText and better accuracy-computation trade-offs on GSM8K. On 16B DMax-LLaDA2.0 models with hybrid masked-uniform diffusion, we increase decoding parallelism with similar or higher accuracy on math and code benchmarks. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.06648 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-06T05:24:20.384Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).