Metraipolis › archive › dc_b503d433d732

Representation-based Masked Diffusion Model — arXiv:2609.12382

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.12382
License (as archived)
CC0-1.0
Archived
2026-10-16T03:08:20.907Z
SHA-256 of the archived text
13a0748149af849c671e1b7f535347051c12a464312e10b29ccc379eafa74dc1
Stable URL of this page
https://metraipolis.com/documento/dc_b503d433d732

How to cite

Representation-based Masked Diffusion Model — arXiv:2609.12382 — arXiv (en), https://arxiv.org/abs/2609.12382 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b503d433d732 (Metraipolis archive dc_b503d433d732, sha256 13a0748149af849c671e1b7f535347051c12a464312e10b29ccc379eafa74dc1)
BibTeX
@misc{dcb503d433d732,
  title = {Representation-based Masked Diffusion Model — arXiv:2609.12382},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.12382},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b503d433d732; sha256 13a0748149af849c671e1b7f535347051c12a464312e10b29ccc379eafa74dc1},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.12382. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Masked Diffusion Models (MDMs) have emerged as a compelling paradigm for language modeling, offering the capability for efficient parallel text generation. However, existing parallel sampling methods typically update multiple masked tokens independently and ignore the complex mutual dependencies among the masked tokens. This independent updating mechanism lacks global coordination and might lead to incoherent outputs. To address this limitation, we propose Representation-based Masked Diffusion Model (RMDM), a framework that leverages the text representation to explicitly encode global semantics and help to parallel update tokens more precisely. Specifically, we first encode text into a continuous semantic space using a pretrained encoder and learn an invertible transformation that normalizes the representation distribution to a Gaussian prior, facilitating efficient sampling during generation. Conditioned on this latent semantic representation, we train a masked diffusion model to learn the conditional text distribution, where the representation serves as global semantic guidance to coordinate parallel token updates and faithfully approximate the target distribution. Empirical results demonstrate that RMDM significantly improves generation quality, particularly in aggressive few-step sampling regimes.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-14 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2609.12382 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T03:02:47.538Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).