Metraipolis › archive › dc_559a4667014e
CanvasAnneal: Curriculum Reinforcement Learning for Diffusion Language Models — arXiv:2609.13060
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.13060
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T03:15:57.770Z
- SHA-256 of the archived text
bfefb91251c4fe40dc4c3b3c377af4a358c6501361b042087fc8991fcfeb4f6a- Stable URL of this page
https://metraipolis.com/documento/dc_559a4667014e
How to cite
CanvasAnneal: Curriculum Reinforcement Learning for Diffusion Language Models — arXiv:2609.13060 — arXiv (en), https://arxiv.org/abs/2609.13060 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_559a4667014e (Metraipolis archive dc_559a4667014e, sha256 bfefb91251c4fe40dc4c3b3c377af4a358c6501361b042087fc8991fcfeb4f6a)
BibTeX
@misc{dc559a4667014e,
title = {CanvasAnneal: Curriculum Reinforcement Learning for Diffusion Language Models — arXiv:2609.13060},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.13060},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_559a4667014e; sha256 bfefb91251c4fe40dc4c3b3c377af4a358c6501361b042087fc8991fcfeb4f6a},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.13060. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Diffusion Language Models (DLMs) offer promising parallel generation capabilities but lag behind autoregressive models in complex reasoning and tool-use tasks. While Reinforcement Learning (RL) has recently been applied to enhance DLMs, standard RL approaches suffer from an exploration bottleneck. To address this, we inject reasoning priors from a stronger teacher model to guide RL exploration. In this paper, we introduce CanvasAnneal, a curriculum-guided diffusion RL framework. During the initial RL phase, we warm-start exploration by injecting teacher-generated reasoning traces into the initial diffusion canvas. As training progresses, we gradually remove this guidance and require the model to generate more of the reasoning trajectory independently. Across mathematical reasoning and tool-use benchmarks, CanvasAnneal improves over standard diffu-GRPO on MATH500, Countdown, and Tau2 and substantially accelerates reward improvement on several tasks, while gains are task-dependent. Our results suggest that structured training-time guidance can alleviate exploration bottlenecks in diffusion RL and speed up convergence on harder tasks. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-14 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.13060 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T03:05:44.087Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).