Metraipolis › archive › dc_bbc4a56d9d77

Global Context Guidance for Diffusion Large Language Models — arXiv:2611.10577

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.10577
License (as archived)
CC0-1.0
Archived
2026-11-10T18:44:28.392Z
SHA-256 of the archived text
866b445865fe21f5284961a057528192278cda5fd7423b061e24f0558e402b05
Stable URL of this page
https://metraipolis.com/documento/dc_bbc4a56d9d77

How to cite

Global Context Guidance for Diffusion Large Language Models — arXiv:2611.10577 — arXiv (en), https://arxiv.org/abs/2611.10577 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bbc4a56d9d77 (Metraipolis archive dc_bbc4a56d9d77, sha256 866b445865fe21f5284961a057528192278cda5fd7423b061e24f0558e402b05)
BibTeX
@misc{dcbbc4a56d9d77,
  title = {Global Context Guidance for Diffusion Large Language Models — arXiv:2611.10577},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.10577},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bbc4a56d9d77; sha256 866b445865fe21f5284961a057528192278cda5fd7423b061e24f0558e402b05},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.10577. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Diffusion large language models (LLMs) have recently emerged as a promising alternative to autoregressive LLMs, enabling parallel generation through iterative denoising. Recent guidance methods improve diffusion LLMs by introducing a weak model to guide the prediction of the base model during denoising. However, the influence of the weak model during denoising remains less clear, and computing its prediction requires an additional forward pass at each denoising step. We present in this paper an in-depth analysis of how diffusion LLMs use contextual information during denoising. Our analysis reveals that diffusion LLMs rely more heavily on local context while often underutilizing global context. Based on this observation, we introduce Global Context Guidance (GCG), a feature-space guidance method that enhances global-context utilization without constructing an explicit weak model. Extensive experiments on LLaDA-8B, Dream-7B, and LLaDA-1.5 across diverse benchmarks demonstrate that GCG consistently outperforms existing guidance methods, while incurring much lower inference overhead.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2611.10577 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-10T18:37:52.426Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).