Metraipolis › archive › dc_0c70562c482d

Masked diffusion as random sequential adsorption: an exactly solvable model for parallel decoding — arXiv:2611.07845

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.07845
License (as archived)
CC0-1.0
Archived
2026-11-06T14:42:41.518Z
SHA-256 of the archived text
dfc7160a9456ef7c9c3e5fd6d3b2695bcd9d31b3f2c9c0293a48fac8a5652917
Stable URL of this page
https://metraipolis.com/documento/dc_0c70562c482d

How to cite

Masked diffusion as random sequential adsorption: an exactly solvable model for parallel decoding — arXiv:2611.07845 — arXiv (en), https://arxiv.org/abs/2611.07845 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0c70562c482d (Metraipolis archive dc_0c70562c482d, sha256 dfc7160a9456ef7c9c3e5fd6d3b2695bcd9d31b3f2c9c0293a48fac8a5652917)
BibTeX
@misc{dc0c70562c482d,
  title = {Masked diffusion as random sequential adsorption: an exactly solvable model for parallel decoding — arXiv:2611.07845},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.07845},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0c70562c482d; sha256 dfc7160a9456ef7c9c3e5fd6d3b2695bcd9d31b3f2c9c0293a48fac8a5652917},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.07845. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Masked diffusion models generate text by reversing a stochastic masking process. Unlike autoregressive LLMs, which generate text one token at a time, they can reveal many tokens in parallel, at the price of an error, since tokens revealed together are drawn independently. We show that this trade-off maps onto the statistical physics of driven many-particle systems. (i) For any data, the error of parallel generation is the entropy production of the generation process relative to the reversed masking process. (ii) For Markov data, generation is a random sequential adsorption process with labelled gaps, which we solve exactly for any number of steps. (iii) For many steps, the error becomes the dissipation of a slowly driven system: the unmasking schedule plays the role of the driving protocol, the number of steps that of its duration, and the error is set by a thermodynamic metric that we obtain in closed form from the mutual information between tokens. Optimal schedules are geodesics of this metric. The exact solution shows how correlations in the text shape both the error and the quality of the output. The longer the correlations, the more the error concentrates at the start of generation. Schedules that start with small steps then gain more, both in path error and in perplexity, and need fewer steps to reach their asymptotic behaviour.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Statistical Mechanics (cond-mat.stat-mech) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.07845 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-06T14:18:42.691Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).