Metraipolis › archive › dc_73d3db7bbc88

FlowCPO: A Unified Divergence View of Preference Alignment for Flow Models — arXiv:2609.09905

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.09905
License (as archived)
CC0-1.0
Archived
2026-10-16T07:26:37.772Z
SHA-256 of the archived text
557d2d269578526d4f1e97686b9b403e817c7e5b63eb62e5593e72a5d12b71e2
Stable URL of this page
https://metraipolis.com/documento/dc_73d3db7bbc88

How to cite

FlowCPO: A Unified Divergence View of Preference Alignment for Flow Models — arXiv:2609.09905 — arXiv (en), https://arxiv.org/abs/2609.09905 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_73d3db7bbc88 (Metraipolis archive dc_73d3db7bbc88, sha256 557d2d269578526d4f1e97686b9b403e817c7e5b63eb62e5593e72a5d12b71e2)
BibTeX
@misc{dc73d3db7bbc88,
  title = {FlowCPO: A Unified Divergence View of Preference Alignment for Flow Models — arXiv:2609.09905},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.09905},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_73d3db7bbc88; sha256 557d2d269578526d4f1e97686b9b403e817c7e5b63eb62e5593e72a5d12b71e2},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.09905. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Preference alignment for flow and diffusion models now spans online reinforcement learning and offline preference optimization, but the relation between these methods remains unclear. In particular, existing forward-process alignment methods require fresh samples from the current model, while offline methods based on fixed preference pairs rely primarily on positive-only fine-tuning or DPO-style likelihood-ratio surrogates. We organize these approaches through a divergence-based framework and introduce FlowCPO, an offline forward-KL objective that uses both preferred and dispreferred samples without online rollouts. For linear interpolation, we show under explicit regularity conditions that the forward-KL objective is bounded by a contrastive flow matching loss, yielding a tractable surrogate on fixed data. We further show that this loss is nonnegative, whereas the signed regression loss of simplified FlowDPO can be unbounded below. In the in-domain setting, FlowCPO achieves higher mean GenEval and OCR scores than the evaluated baselines, reaching 0.84 and 0.87 versus 0.81 and 0.74 for FlowDPO at CFG 3.0. In the out-of-domain setting, the results are mixed, with the best GenEval result but lower reward scores than RFT on several metrics.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-10 in cs.AI).
Categorie arXiv: Machine Learning (stat.ML) ; Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.09905 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T07:00:16.365Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-10?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).