Metraipolis › archive › dc_61e7873618fa

Improving Few-Step Language Flows with Untied Self-Conditioning — arXiv:2608.22244

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.22244
License (as archived)
CC0-1.0
Archived
2026-10-18T07:42:42.127Z
SHA-256 of the archived text
60069a69cb3be9f88d17e3ee5c538ec53e47f3390de7c9a1689ca3133d987886
Stable URL of this page
https://metraipolis.com/documento/dc_61e7873618fa

How to cite

Improving Few-Step Language Flows with Untied Self-Conditioning — arXiv:2608.22244 — arXiv (en), https://arxiv.org/abs/2608.22244 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_61e7873618fa (Metraipolis archive dc_61e7873618fa, sha256 60069a69cb3be9f88d17e3ee5c538ec53e47f3390de7c9a1689ca3133d987886)
BibTeX
@misc{dc61e7873618fa,
  title = {Improving Few-Step Language Flows with Untied Self-Conditioning — arXiv:2608.22244},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.22244},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_61e7873618fa; sha256 60069a69cb3be9f88d17e3ee5c538ec53e47f3390de7c9a1689ca3133d987886},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.22244. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Flow-matching language models refine all token positions in parallel and can trade sampling steps for latency, yet generation quality still degrades sharply with few sampling steps. We trace a source of this degradation to a train--inference mismatch in previous-prediction self-conditioning: during training, the self-conditioning input is computed from the current noisy state with no intervening solver step; during sampling, the solver folds the previous prediction into the latent before that same prediction reappears as the explicit self-conditioning input. This coupling, absent during training, creates redundancy that grows with step width. We show that the mismatch degrades both the self-conditioning input and the solver update, and derive a correction for each from the model's own structure. From the frozen projection weights we identify directions along which the self-conditioning input is redundant with the latent and dampen them; from the solver's integration structure we derive that a step-average prediction is needed and approximate it from prediction history, with scale set by offline trajectory statistics. The resulting sampler, Untied Self-Conditioning, requires no retraining and uses one evaluation per step. At 8 sampling steps on LangFlow, it reduces OpenWebText generative perplexity from $531$ to~$62$ ($8.6\times$); under an adapted Arena-Hard-Auto~v2 protocol, its outputs are preferred in $96\%$ of pairwise comparisons. On ELF-B it reduces generative perplexity from $71$ to~$43$. Improvements hold from 8 to 256 sampling steps.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-25 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.22244 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T07:37:52.863Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).