Metraipolis › archive › dc_2d78e01a8cc8

DUET: Dual-Sparse Inference for Block Diffusion Language Models — arXiv:2611.09239

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.09239
License (as archived)
CC0-1.0
Archived
2026-11-09T22:35:58.649Z
SHA-256 of the archived text
ca14e814e99ba2c385b98721c7f05c9aef346ed7c23526207aa454bd1e2226ae
Stable URL of this page
https://metraipolis.com/documento/dc_2d78e01a8cc8

How to cite

DUET: Dual-Sparse Inference for Block Diffusion Language Models — arXiv:2611.09239 — arXiv (en), https://arxiv.org/abs/2611.09239 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2d78e01a8cc8 (Metraipolis archive dc_2d78e01a8cc8, sha256 ca14e814e99ba2c385b98721c7f05c9aef346ed7c23526207aa454bd1e2226ae)
BibTeX
@misc{dc2d78e01a8cc8,
  title = {DUET: Dual-Sparse Inference for Block Diffusion Language Models — arXiv:2611.09239},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.09239},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2d78e01a8cc8; sha256 ca14e814e99ba2c385b98721c7f05c9aef346ed7c23526207aa454bd1e2226ae},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.09239. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Diffusion language models (DLMs) enable parallel token generation through iterative denoising, yet their inference remains costly due to repeated attention over a growing historical prefix and repeated processing of the entire current generation block. Historical attention becomes increasingly expensive as generation progresses, while all current-block positions are propagated through the Transformer at every denoising step even though only a small fraction are typically resolved at that step. In this paper, we introduce Duet, a training-free dual-sparse inference framework that addresses these challenges through prefix Key-Value (KV) sparsity and early query sparsity, respectively. Specifically, Duet exploits the concentration and cross-step stability of historical attention within each generation block to construct a compact historical KV subset that is reused across denoising steps. For query sparsity, Duet predicts update positions from shallow representations and propagates only the corresponding queries through the suffix Transformer layers, avoiding deep computation at positions unlikely to be updated in the current step. Experiments show that Duet maintains competitive accuracy while achieving 1.34$\times$/2.83$\times$ end-to-end/per-step speedups on LLaDA2.1-mini and 2.41$\times$/4.72$\times$ on SDAR-8B-Chat-b32 at a 32K context length.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2611.09239 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-09T22:31:01.947Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).