Metraipolis › archive › dc_3c5dacefe176

Retimed Bellman Flows: Escaping the Impossible Triangle of Velocity Bootstrapping — arXiv:2609.32828

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.32828
License (as archived)
CC0-1.0
Archived
2026-10-07T22:36:53.061Z
SHA-256 of the archived text
6542247ecbb94c42e2f0b09b1fb1b4d1ebddba9d31ec402cc46f3377588524bf
Stable URL of this page
https://metraipolis.com/documento/dc_3c5dacefe176

How to cite

Retimed Bellman Flows: Escaping the Impossible Triangle of Velocity Bootstrapping — arXiv:2609.32828 — arXiv (en), https://arxiv.org/abs/2609.32828 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3c5dacefe176 (Metraipolis archive dc_3c5dacefe176, sha256 6542247ecbb94c42e2f0b09b1fb1b4d1ebddba9d31ec402cc46f3377588524bf)
BibTeX
@misc{dc3c5dacefe176,
  title = {Retimed Bellman Flows: Escaping the Impossible Triangle of Velocity Bootstrapping — arXiv:2609.32828},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.32828},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3c5dacefe176; sha256 6542247ecbb94c42e2f0b09b1fb1b4d1ebddba9d31ec402cc46f3377588524bf},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32828. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Flow critics learn return distributions by transporting Gaussian noise to Bellman endpoints via continuous velocity fields. While velocity bootstrapping stabilizes training by querying a successor teacher, existing methods face a structural dilemma: on straight paths, no residual-free same-time affine mapping can preserve Gaussian initial noise while maintaining an unbiased target. To overcome this limitation, we introduce Retimed Bellman Flows (ReBF). ReBF queries the teacher critic at a dynamically shifted earlier flow time, aligning intermediate student and teacher trajectories. By combining this retimed clock with fresh, decoupled noise generation, ReBF constructs a provably conditionally unbiased velocity target that preserves the Bellman fixed point and contracts under Wasserstein distances. Empirically, ReBF reduces $W_1$ distance to ground-truth return distributions by up to $7.7\times$ on synthetic MRPs and outperforms existing flow critics across 38 challenging OGBench and D4RL offline reinforcement learning tasks.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.32828 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T22:14:27.778Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).