Metraipolis › archive › dc_bffb85ec4ea3

WTF?! Simulation-Free Reinforcement Learning with Wasserstein-Tilted Flow Maps — arXiv:2609.27033

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.27033
License (as archived)
CC0-1.0
Archived
2026-10-08T10:52:04.549Z
SHA-256 of the archived text
fe853b742b89c6774fa5cf13594770fbf6b2c95f9c4dfd9c318294dc955f977e
Stable URL of this page
https://metraipolis.com/documento/dc_bffb85ec4ea3

How to cite

WTF?! Simulation-Free Reinforcement Learning with Wasserstein-Tilted Flow Maps — arXiv:2609.27033 — arXiv (en), https://arxiv.org/abs/2609.27033 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bffb85ec4ea3 (Metraipolis archive dc_bffb85ec4ea3, sha256 fe853b742b89c6774fa5cf13594770fbf6b2c95f9c4dfd9c318294dc955f977e)
BibTeX
@misc{dcbffb85ec4ea3,
  title = {WTF?! Simulation-Free Reinforcement Learning with Wasserstein-Tilted Flow Maps — arXiv:2609.27033},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.27033},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bffb85ec4ea3; sha256 fe853b742b89c6774fa5cf13594770fbf6b2c95f9c4dfd9c318294dc955f977e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.27033. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Reward fine-tuning aims to update a pre-trained flow-based generative model to improve the downstream reward of its generated samples. Existing methods typically formulate this problem as sampling from a reward-tilted distribution, the solution to a KL-regularized reward-maximization problem. Here, we introduce an optimal transport regularizer built directly from the pre-trained drift. Unlike KL reward tilting, the resulting objective transports individual samples toward higher reward rather than reweighting the base distribution. We show that the resulting problem is equivalent to a deterministic optimal control problem on the flow. Given a pre-trained flow map, this equivalence yields a simulation-free reinforcement learning algorithm for fine-tuning generative flows. We call the resulting framework Wasserstein-Tilted Flow Maps (WTF), the first end-to-end fine-tuning recipe native to flow maps. The output is a fine-tuned flow map that retains strong reward-aligned performance at few-step inference budgets without post-hoc distillation. Experiments on ImageNet-256 and text-to-image show that WTF achieves higher reward with comparable or higher diversity than baselines, while requiring up to $280\times$ less training compute. More broadly, we argue that accelerated samplers such as flow maps are essential infrastructure for efficient post-training, and that the dominant KL-regularized formulation is only one of many choices worth revisiting.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-24 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Computer Vision and Pattern Recognition (cs.CV); Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2609.27033 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T10:06:16.697Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).