Metraipolis › archive › dc_98d6f48fa784

Register Tokens for Bounded-State Reasoning in Diffusion Language Models — arXiv:2609.16372

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.16372
License (as archived)
CC0-1.0
Archived
2026-10-15T19:19:15.866Z
SHA-256 of the archived text
3f2ff4d3c42a50e15ddeb9045c84b0943a32b6cd6286d08c290f4a13bca94476
Stable URL of this page
https://metraipolis.com/documento/dc_98d6f48fa784

How to cite

Register Tokens for Bounded-State Reasoning in Diffusion Language Models — arXiv:2609.16372 — arXiv (en), https://arxiv.org/abs/2609.16372 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_98d6f48fa784 (Metraipolis archive dc_98d6f48fa784, sha256 3f2ff4d3c42a50e15ddeb9045c84b0943a32b6cd6286d08c290f4a13bca94476)
BibTeX
@misc{dc98d6f48fa784,
  title = {Register Tokens for Bounded-State Reasoning in Diffusion Language Models — arXiv:2609.16372},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.16372},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_98d6f48fa784; sha256 3f2ff4d3c42a50e15ddeb9045c84b0943a32b6cd6286d08c290f4a13bca94476},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.16372. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Masked diffusion language models (dLLMs) generate text by iteratively denoising masked tokens with bidirectional attention. Extending reasoning across generation chunks normally requires keeping earlier generated text in context. We ask whether a dLLM can instead continue reasoning after that text is cleared, using only a fixed-size carried state. We implement this state as a small number of register tokens: dedicated fixed-position tokens whose continuous hidden states are trained to carry reasoning progress across generation chunks. We post-train dLLMs to decode a chunk of text, clear it while preserving the register values, and continue decoding from the prompt and carried state. In our main comparisons on LLaDA and Dream, registers outperform discrete-text carry on every benchmark, with gains of up to 8.5 points on math and 19.5 points on code. Registers are especially effective for bounded code generation, where correct programs usually span several chunks. Finally, registers can be further refined with reinforcement learning on long-horizon reasoning tasks.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-16 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.16372 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T18:32:32.805Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-16?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).