Metraipolis › archive › dc_fe0b0628445e

ELF-REG: Scaling Continuous Diffusion Language Models to Reasoning Tasks — arXiv:2609.29102

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.29102
License (as archived)
CC0-1.0
Archived
2026-10-08T04:49:25.804Z
SHA-256 of the archived text
af83adc2998f2815a7eaad738e4b3fc564fbc24efc09f5b2859069516f8637b9
Stable URL of this page
https://metraipolis.com/documento/dc_fe0b0628445e

How to cite

ELF-REG: Scaling Continuous Diffusion Language Models to Reasoning Tasks — arXiv:2609.29102 — arXiv (en), https://arxiv.org/abs/2609.29102 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fe0b0628445e (Metraipolis archive dc_fe0b0628445e, sha256 af83adc2998f2815a7eaad738e4b3fc564fbc24efc09f5b2859069516f8637b9)
BibTeX
@misc{dcfe0b0628445e,
  title = {ELF-REG: Scaling Continuous Diffusion Language Models to Reasoning Tasks — arXiv:2609.29102},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.29102},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fe0b0628445e; sha256 af83adc2998f2815a7eaad738e4b3fc564fbc24efc09f5b2859069516f8637b9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.29102. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Fully continuous diffusion language models (dLMs) denoise continuous representations without intermediate discretization, then decode all response tokens in parallel at the final step. Their performance on challenging reasoning tasks remains less established than that of autoregressive (AR) LLMs and masked dLMs. We scale Embedded Language Flows (ELF) to mathematical reasoning and code generation on GSM8K, MATH-500, HumanEval, and MBPP. We introduce ELF-REG, which improves learning with representation alignment and entanglement (REPA+REG), where a frozen AR teacher supervises intermediate denoiser features and supplies a global representation that is jointly denoised with the response. ELF-REG-L achieves 55.96% pass@1 on GSM8K at 64 network function evaluations (NFE), and 13.39% on MATH-500 and 22.56% on HumanEval at 128 NFE. It outperforms the evaluated comparable-scale dLMs in pass@1 on GSM8K and code, and improves MATH-500 pass@1 from 10.55% for the ELF-L baseline to 13.39% with ELF-REG-L. Without few-step training, the same task-specific checkpoints support strong low-NFE performance through early-stop, which decodes an intermediate clean prediction without completing the denoising trajectory. At 16 NFE, ELF-REG-L reaches 41.21% HumanEval pass@10, outperforming recent continuous dLMs of comparable scale.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-25 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.29102 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T04:27:20.032Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).