Metraipolis › archive › dc_8403fc0b6d10

CForce: Boosting Parallel Decoding for dLLMs via Consistency Forcing — arXiv:2608.13925

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.13925
License (as archived)
CC0-1.0
Archived
2026-10-19T02:21:12.831Z
SHA-256 of the archived text
7f7df511042f55db28f0544660b0b81ca92a952b1c70827baee8b18fb8041fd8
Stable URL of this page
https://metraipolis.com/documento/dc_8403fc0b6d10

How to cite

CForce: Boosting Parallel Decoding for dLLMs via Consistency Forcing — arXiv:2608.13925 — arXiv (en), https://arxiv.org/abs/2608.13925 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8403fc0b6d10 (Metraipolis archive dc_8403fc0b6d10, sha256 7f7df511042f55db28f0544660b0b81ca92a952b1c70827baee8b18fb8041fd8)
BibTeX
@misc{dc8403fc0b6d10,
  title = {CForce: Boosting Parallel Decoding for dLLMs via Consistency Forcing — arXiv:2608.13925},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.13925},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8403fc0b6d10; sha256 7f7df511042f55db28f0544660b0b81ca92a952b1c70827baee8b18fb8041fd8},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.13925. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Diffusion large language models (dLLMs) accelerate language generation by predicting multiple masks in a single forward pass. However, existing dLLMs can suffer from unreliable predictions in early denoising stages under aggressive parallelism strategies, leading to errors that can propagate to later stages. To tackle this issue, we present Consistency Forcing (CForce) for dLLMs, a distillation method to force the mask predictions of early stages to align with those of later stages. CForce trains the model on pre-collected self-rollout trajectories, thereby improving training-inference alignment. We introduce Confidence Adaptive KL Divergence as a distillation objective to conjoin the merits of forward and reverse KL. We further provide a theoretical analysis for the consistency objective to explain why CForce can approximately minimize the prediction error of early stages. Critically, the same formulation applies to both mask-to-token decoding and edit-capable decoding; in the edit-capable case, later token-to-token refinements provide additional supervision for earlier masked-state predictions. Experiments on non-edit and edit-capable LLaDA models show improved speed-quality trade-offs, especially under high-parallelism decoding budgets. Code is available at: this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-17 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.13925 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T02:02:02.885Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).