Metraipolis › archive › dc_31f63db42012

Test-Time Scaling for Diffusion Language Models with Frequency-Aware Remasking — arXiv:2611.10486

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.10486
License (as archived): CC0-1.0
Archived: 2026-11-10T18:42:32.439Z
SHA-256 of the archived text: d25a422732e653b7f8cfbce9a1f42ff0279a90db3844c2959875f6b3c2ec0c6f
Stable URL of this page: https://metraipolis.com/documento/dc_31f63db42012

How to cite

Test-Time Scaling for Diffusion Language Models with Frequency-Aware Remasking — arXiv:2611.10486 — arXiv (en), https://arxiv.org/abs/2611.10486 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_31f63db42012 (Metraipolis archive dc_31f63db42012, sha256 d25a422732e653b7f8cfbce9a1f42ff0279a90db3844c2959875f6b3c2ec0c6f)

BibTeX

@misc{dc31f63db42012,
  title = {Test-Time Scaling for Diffusion Language Models with Frequency-Aware Remasking — arXiv:2611.10486},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.10486},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_31f63db42012; sha256 d25a422732e653b7f8cfbce9a1f42ff0279a90db3844c2959875f6b3c2ec0c6f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.10486. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Diffusion language models (DLMs) generate text through iterative denoising, where decisions about which tokens to commit or remask strongly affect output quality. Existing certainty-aware strategies rely on instantaneous confidence or entropy, but these local signals can be misleading on difficult reasoning problems. We formulate DLM test-time scaling as adaptive allocation of both inference compute and decoding strategies. Our two-stage approach uses a fast base sampler to solve easy questions, then applies frequency-aware remasking to the remaining hard questions. This strategy tracks how often each token's prediction changes across denoising steps and preferentially commits frequently changing tokens, encouraging samples to explore alternative reasoning paths rather than collapse onto the model's preferred trajectory. Experiments with LLaDA-8B-Instruct and Dream-7B-Instruct show consistent improvements over a strong adaptive allocation baseline, with absolute accuracy gains of up to 3.73 percentage points (LLaDA) and 7.33 percentage points (Dream) on MATH-500, and absolute coverage gains of up to 11.39 percentage points (LLaDA) and 10.98 percentage points (Dream) on HumanEval. On questions unsolved by the base certainty-aware strategy after 100 samples, frequency-aware remasking recovers solutions for 3.75% to 18.75% of examples.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.10486 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-10T18:37:37.591Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).