Metraipolis › archive › dc_3c46fb40ee20
On-Policy Residual Learning for Diffusion Language Models — arXiv:2610.20687
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.20687
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T10:19:59.598Z
- SHA-256 of the archived text
0d8d30de99c275137101b0d1bb8e1e9872c5aa7bbc37008b40308a832182162f- Stable URL of this page
https://metraipolis.com/documento/dc_3c46fb40ee20
How to cite
On-Policy Residual Learning for Diffusion Language Models — arXiv:2610.20687 — arXiv (en), https://arxiv.org/abs/2610.20687 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3c46fb40ee20 (Metraipolis archive dc_3c46fb40ee20, sha256 0d8d30de99c275137101b0d1bb8e1e9872c5aa7bbc37008b40308a832182162f)
BibTeX
@misc{dc3c46fb40ee20,
title = {On-Policy Residual Learning for Diffusion Language Models — arXiv:2610.20687},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.20687},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3c46fb40ee20; sha256 0d8d30de99c275137101b0d1bb8e1e9872c5aa7bbc37008b40308a832182162f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.20687. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Residual context is key to unlocking the speed-quality Pareto frontier of diffusion language models (DLMs) by carrying forward predictive distributions across denoising steps. Yet turning this seemingly simple mechanism into an effective learning paradigm is far from simple: there is no clear recipe for how residual context should be learned. We introduce ORL, a framework for on-policy residual learning in DLMs. Algorithmically, we identify that residual learning is inherently on-policy. ORL learns from model-induced residual trajectories on the fly and introduces an entropy-contraction objective that sharpens predictions as the model revisits its own predictions. System-wise, exact residual supervision requires capturing, retaining, and reprocessing many trajectory-dependent states. ORL co-designs the algorithm with asynchronous residual capture and transport, specialized block-diffusion attention, and packed compute-balanced training, making it practical at scale. On a 4B DLM converted from Qwen3-4B, ORL improves tokens per forward (TPF) by up to 6.2x and, with our dlmrt engine, end-to-end throughput by approximately 6.9x at comparable or better quality. In single-request serving on one H100, this mode achieves a stronger speed-quality Pareto frontier than autoregressive models with EAGLE3 and DFlash speculative decoding under both strict and relaxed verification, running 1.74x faster than DFlash at matched accuracy. ORL also transfers to existing DLMs: on LLaDA-2.1-mini, 36 H100-hours and 4.2M self-generated tokens, about 10x less data than prior LLaDA recipes, yield up to 1.5x higher TPF and up to 3x higher throughput at comparable quality, and on DiffusionGemma, ORL raises TPF by 1.1-1.2x. With dlmrt and NVFP4 quantization, ORL models decode a single request at up to 5,046 tokens/s on one B200, 7,708 on eight B200s, and 2,788 on a consumer RTX 5090, with comparable quality. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.20687 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T08:48:02.647Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).