Metraipolis › archive › dc_ae4bb2126729

Large Flow Language Models — arXiv:2610.28559

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.28559
License (as archived)
CC0-1.0
Archived
2026-10-21T07:08:36.498Z
SHA-256 of the archived text
79a59ace77e07a41dd365c35b2edf31bb500bc4c846d885627e23b1bf1d1d9b9
Stable URL of this page
https://metraipolis.com/documento/dc_ae4bb2126729

How to cite

Large Flow Language Models — arXiv:2610.28559 — arXiv (en), https://arxiv.org/abs/2610.28559 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ae4bb2126729 (Metraipolis archive dc_ae4bb2126729, sha256 79a59ace77e07a41dd365c35b2edf31bb500bc4c846d885627e23b1bf1d1d9b9)
BibTeX
@misc{dcae4bb2126729,
  title = {Large Flow Language Models — arXiv:2610.28559},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.28559},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ae4bb2126729; sha256 79a59ace77e07a41dd365c35b2edf31bb500bc4c846d885627e23b1bf1d1d9b9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.28559. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Continuous flow language models offer a path to parallel generation with tools for control and acceleration unavailable to discrete diffusion, yet effective techniques for building them at the billion-parameter scale remain less established. We present Lafa (Large flow language model), a fast, high-performing flow language model scaled to 8B parameters in a data-efficient manner. Lafa is a blockwise flow model with causal attention on the prefix and bidirectional, asynchronous flow within each block. This design enables efficient training by adapting pretrained autoregressive (AR) models and fast inference via prefix caching and blockwise parallel decoding. We train Lafa at 0.5B, 7B, and 8B scales with fewer than 30B tokens, more efficient than previous diffusion models, and evaluate it on instruction following, reasoning, and coding, where the learned flow matches or surpasses the quality of AR and can be distilled into flow maps for one- and few-step generation. Notably, flow training preserves or even improves pretrained AR capabilities via a multi-token prediction effect, which we leverage in a self-speculative decoding scheme that verifies flow-based drafts autoregressively within a single model. This yields 2.54x the throughput of an AR model at comparable quality, outperforming multi-token prediction and discrete diffusion methods. Our findings show that continuous flows can serve as a scalable foundation for language modeling.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.28559 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-21T07:05:43.668Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).