Metraipolis › archive › dc_ffae93d2fbc5

BYOD: Build Your Own DLM - Efficient Autoregressive-to-Diffusion Language Model Conversion — arXiv:2610.41574

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.41574
License (as archived): CC0-1.0
Archived: 2026-11-03T07:45:29.522Z
SHA-256 of the archived text: b2224af4975e44b3e48b1ecc0c33c0ccd5dfd2ebe4fdd2b6b31041bb29e3dcf3
Stable URL of this page: https://metraipolis.com/documento/dc_ffae93d2fbc5

How to cite

BYOD: Build Your Own DLM - Efficient Autoregressive-to-Diffusion Language Model Conversion — arXiv:2610.41574 — arXiv (en), https://arxiv.org/abs/2610.41574 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ffae93d2fbc5 (Metraipolis archive dc_ffae93d2fbc5, sha256 b2224af4975e44b3e48b1ecc0c33c0ccd5dfd2ebe4fdd2b6b31041bb29e3dcf3)

BibTeX

@misc{dcffae93d2fbc5,
  title = {BYOD: Build Your Own DLM - Efficient Autoregressive-to-Diffusion Language Model Conversion — arXiv:2610.41574},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.41574},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ffae93d2fbc5; sha256 b2224af4975e44b3e48b1ecc0c33c0ccd5dfd2ebe4fdd2b6b31041bb29e3dcf3},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41574. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Unlike autoregressive (AR) models, diffusion language models (DLMs) permit parallel token updates, flexible generation, and bidirectional attention, but leading systems rely on costly pretraining or full-model fine-tuning. We present Build Your Own DLM (BYOD), a generalizable LoRA-only recipe for parameter-efficient AR-to-DLM conversion. Without architecture-specific tuning, BYOD induces diffusion generation in Gemma 2 9B, Llama 3.1 8B, Qwen2.5 7B, and Ministral 8B; to our knowledge, this is the first reported AR-to-DLM conversion of a Mistral model. The adapters comprise 4.1-5.8% of final parameters while every base-model parameter remains frozen; each 25,000-iteration conversion needs at most 102.4M tokens, processed in 3.1-4.0 hours on one Google Colab GPU. All four models generate from fully masked sequences, use bidirectional context in a controlled attention test, and retain substantial parent-model knowledge. BYOD-Gemma is the strongest BYOD conversion on our nine-task evaluation, reaching 50.6% average accuracy compared with 51.9% for LLaDA 8B Instruct in the same harness. Open-generation quality at low network function evaluation (NFE) budgets remains the principal limitation: BYOD perplexity is 8.3-11.1 versus 4.1 for LLaDA and 2.2-3.0 for the AR parents when generating 128 tokens in 64 steps. To enable further experimentation and inspection, we provide a standalone training and evaluation implementation and interactive demos for all four conversions. BYOD thus provides a practical route for extending diffusion generation to other existing model families while identifying the gap that efficient conversion must next close.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.41574 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T07:39:52.175Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).