Metraipolis › archive › dc_d75804f1cdf3
Omni-Diffusion-Distill: Few-Step Distillation of Unified Multimodal Diffusion Large Language Models — arXiv:2610.10990
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.10990
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T06:23:57.665Z
- SHA-256 of the archived text
ce2f71ee15286246cbae301949a1ee53948ba6f87c9ee0c6c5f8744a33c730ff- Stable URL of this page
https://metraipolis.com/documento/dc_d75804f1cdf3
How to cite
Omni-Diffusion-Distill: Few-Step Distillation of Unified Multimodal Diffusion Large Language Models — arXiv:2610.10990 — arXiv (en), https://arxiv.org/abs/2610.10990 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d75804f1cdf3 (Metraipolis archive dc_d75804f1cdf3, sha256 ce2f71ee15286246cbae301949a1ee53948ba6f87c9ee0c6c5f8744a33c730ff)
BibTeX
@misc{dcd75804f1cdf3,
title = {Omni-Diffusion-Distill: Few-Step Distillation of Unified Multimodal Diffusion Large Language Models — arXiv:2610.10990},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.10990},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d75804f1cdf3; sha256 ce2f71ee15286246cbae301949a1ee53948ba6f87c9ee0c6c5f8744a33c730ff},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.10990. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Unified multimodal diffusion large language models (dLLMs) offer a single architecture for both image generation and multimodal understanding, but their iterative decoding requires tens to hundreds of forward passes. Existing few-step distillation methods largely focus on either image generation or text generation, making it unclear how to compress a fully discrete multimodal dLLM into a single efficient student while preserving both generation and understanding. We introduce Omni-Diffusion-Distill, a unified two-stage distillation framework that retains strong generation and understanding capabilities while substantially reducing the inference cost of a unified multimodal dLLM. Omni-Diffusion-Distill aligns the distillation of both generation and understanding, for both images and text, in the discrete token space. In the first stage, the student is trained to skip decoding steps by replaying cached teacher trajectories, and in the second stage the student is refined on intermediate states along its own rollouts. We further remedy two sources of degradation in unified distillation with a pairwise collision penalty that reduces repetition under parallel text decoding, and entropy-matched guidance that prevents entropy collapse caused by fitting the sharpened teacher distribution in image generation. Omni-Diffusion-Distill achieves state-of-the-art trade-offs between decoding efficiency and generation and understanding performance for multimodal dLLMs, reducing image generation from 128 to 8 decoding steps and multimodal understanding from 512 to 64, giving 18.2x and 21.2x wall-clock speedups. Under these budgets, it scores 0.828 on GenEval and 83.0 on DPG-Bench for text-to-image generation, while reaching GPT judge scores of 20.0 on MM-Vet and 57.2 on COCO captioning (twice the teacher's 28.4 at the same steps) for multimodal understanding. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.10990 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T05:50:31.911Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).