Metraipolis › archive › dc_48a7bf0de5ac

Backpropagating Verifier Rewards Through Continuous Flow Language Models — arXiv:2610.19080

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.19080
License (as archived)
CC0-1.0
Archived
2026-10-15T14:14:56.936Z
SHA-256 of the archived text
ccf15715b49427500ee1e0a357b4fa6a88fcbabf2dc8046e1c72b38a9a8ebec2
Stable URL of this page
https://metraipolis.com/documento/dc_48a7bf0de5ac

How to cite

Backpropagating Verifier Rewards Through Continuous Flow Language Models — arXiv:2610.19080 — arXiv (en), https://arxiv.org/abs/2610.19080 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_48a7bf0de5ac (Metraipolis archive dc_48a7bf0de5ac, sha256 ccf15715b49427500ee1e0a357b4fa6a88fcbabf2dc8046e1c72b38a9a8ebec2)
BibTeX
@misc{dc48a7bf0de5ac,
  title = {Backpropagating Verifier Rewards Through Continuous Flow Language Models — arXiv:2610.19080},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.19080},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_48a7bf0de5ac; sha256 ccf15715b49427500ee1e0a357b4fa6a88fcbabf2dc8046e1c72b38a9a8ebec2},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.19080. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Reinforcement learning (RL) has substantially improved the reasoning of autoregressive and discrete-diffusion language models. Language models that generate continuous token embeddings are emerging as a strong alternative, but RL post-training for them remains understudied. We propose \method{}, a post-training framework for continuous flow language models. Such a model denoises token embeddings over multiple steps and decodes them into discrete tokens only at the last step. To obtain pathwise gradients, which backpropagate through continuous actions and have much lower variance than REINFORCE, we introduce a critic that estimates the value of intermediate token embeddings, bypassing the non-differentiable decoding step. The critic must also keep track of the value function as the policy changes; to this end, we design a critic that predicts a value from every token representation and improves policy optimization. Across four reasoning tasks, \method{} reliably improves a continuous flow language model through RL post-training and outperforms baseline methods of the same type.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.19080 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T13:53:35.520Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).