Metraipolis › archive › dc_b4e91f36ae4f
PreferenceFlow: Test-Time Guidance of Flow-Matching Robot Policies from Human Interventions — arXiv:2609.36872
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.36872
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T19:15:24.518Z
- SHA-256 of the archived text
bb105440643e3defa33af056929aed4ede0ea1c120efeff544c88e6411bb9c19- Stable URL of this page
https://metraipolis.com/documento/dc_b4e91f36ae4f
How to cite
PreferenceFlow: Test-Time Guidance of Flow-Matching Robot Policies from Human Interventions — arXiv:2609.36872 — arXiv (en), https://arxiv.org/abs/2609.36872 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b4e91f36ae4f (Metraipolis archive dc_b4e91f36ae4f, sha256 bb105440643e3defa33af056929aed4ede0ea1c120efeff544c88e6411bb9c19)
BibTeX
@misc{dcb4e91f36ae4f,
title = {PreferenceFlow: Test-Time Guidance of Flow-Matching Robot Policies from Human Interventions — arXiv:2609.36872},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.36872},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b4e91f36ae4f; sha256 bb105440643e3defa33af056929aed4ede0ea1c120efeff544c88e6411bb9c19},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.36872. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Flow-matching policies can represent complex robot behaviors but remain susceptible to local errors under distribution shift at deployment. Many reinforcement learning approaches to policy improvement require reward signals that are difficult to specify or obtain in real-world manipulation. We present PreferenceFlow, a framework for improving a pretrained flow policy at test time without environment rewards or updates to the base policy. Human intervention chunks are paired with robot chunks generated from the same initial conditioning state to train a preference model. During infer- ence, we adopt the QGF sampling update, replacing its value gradient with the preference gradient evaluated at an estimated clean action. A gradient-cap loss penalizes excessive gradients on intervention pairs, while a zero-gradient loss discourages guidance near actions from expert demonstrations. On four real-world precision insertion tasks with a Franka robot, Pref- erenceFlow achieves a mean success rate of 90.5%, compared with 69% for the frozen policy. Ablations support the roles of gradient regularization and correctly ordered preference labels in the evaluated settings. These results demonstrate the utility of human interventions as local preference supervision for guiding frozen generative robot policies. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-30 in cs.RO). Categorie arXiv: Robotics (cs.RO). Pagina del lavoro: https://arxiv.org/abs/2609.36872 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T18:40:59.460Z dalla pagina https://arxiv.org/catchup/cs.RO/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).