Metraipolis › archive › dc_385ac9ba2b05
BCT: Bayes-Consistent Tempering for Flow Language Models — arXiv:2610.27641
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.27641
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-21T07:07:58.143Z
- SHA-256 of the archived text
3bab5ce9b587be7fd491634ed47abe397cf7e18f8c744b092438505dfa3c71ce- Stable URL of this page
https://metraipolis.com/documento/dc_385ac9ba2b05
How to cite
BCT: Bayes-Consistent Tempering for Flow Language Models — arXiv:2610.27641 — arXiv (en), https://arxiv.org/abs/2610.27641 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_385ac9ba2b05 (Metraipolis archive dc_385ac9ba2b05, sha256 3bab5ce9b587be7fd491634ed47abe397cf7e18f8c744b092438505dfa3c71ce)
BibTeX
@misc{dc385ac9ba2b05,
title = {BCT: Bayes-Consistent Tempering for Flow Language Models — arXiv:2610.27641},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.27641},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_385ac9ba2b05; sha256 3bab5ce9b587be7fd491634ed47abe397cf7e18f8c744b092438505dfa3c71ce},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.27641. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Temperature scaling is typically applied as a post-hoc transformation of model probabilities. In posterior-induced flow language models, this creates two mismatches. First, posterior tempering implicitly changes both the endpoint distribution and the distribution of intermediate noisy states, while the denoiser remains trained on the original states. Second, the tempered posterior weights are combined with transport dynamics derived for the untempered flow. These mismatches are reflected empirically: lowering temperature improves GSM8K performance much less reliably in flow LMs than in discrete diffusion LMs. We introduce Bayes-Consistent Tempering (BCT), which makes temperature explicit in both the distributions and dynamics of the generative flow. Its learned realization trains the denoiser on tempered intermediate states and tempers the endpoint distribution using the corresponding tempered dynamics. We also derive training-free variants for conventionally trained checkpoints. Across hyperspherical flow LMs with two denoiser architectures and a von Mises--Fisher flow LM, BCT improves GSM8K generation across broad temperature ranges and achieves the strongest tempering performance. The gains also extend to Sudoku, demonstrating benefits on tightly constrained generation. At very low temperatures, the learned realization is also markedly more robust than the training-free variants, supporting temperature as a property of the generative flow across training and inference rather than a post-hoc probability adjustment. Code and model weights are available at this https URL . Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.27641 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-21T07:05:43.586Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).