Metraipolis › archive › dc_84cbb5a7b4dd

Timestep Weighting: A Hidden Key to Effective ELBO-Based Flow-Matching RL — arXiv:2609.32665

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.32665
License (as archived)
CC0-1.0
Archived
2026-10-07T21:00:31.248Z
SHA-256 of the archived text
b9c4ca117bec36648b89808f80056512e56f506388ea48dd9cde618da30eb971
Stable URL of this page
https://metraipolis.com/documento/dc_84cbb5a7b4dd

How to cite

Timestep Weighting: A Hidden Key to Effective ELBO-Based Flow-Matching RL — arXiv:2609.32665 — arXiv (en), https://arxiv.org/abs/2609.32665 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_84cbb5a7b4dd (Metraipolis archive dc_84cbb5a7b4dd, sha256 b9c4ca117bec36648b89808f80056512e56f506388ea48dd9cde618da30eb971)
BibTeX
@misc{dc84cbb5a7b4dd,
  title = {Timestep Weighting: A Hidden Key to Effective ELBO-Based Flow-Matching RL — arXiv:2609.32665},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.32665},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_84cbb5a7b4dd; sha256 b9c4ca117bec36648b89808f80056512e56f506388ea48dd9cde618da30eb971},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32665. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

ELBO-based reinforcement learning offers a sampler-agnostic approach to fine-tuning flow matching models with reward feedback. Timestep weighting in ELBO-based RL has large impact on performance, and it also provides a unified view (as we show in this work) to understand prediction losses heuristically chosen in prior work, yet it remains under-researched and is often chosen to inherit pretrain configs. We investigate impacts and dynamics of timestep weighting in ELBO-based RL. We show that effective weighting depends on both the reward landscape and stage of learning. (1) Through experiments on controlled CIFAR image generation, complemented by robotics, we investigate how weighting impacts reward-driven updates across noise levels. (2) Through gradient analysis, we reveal distinct patterns of cross-noise coordination across tasks and their evolution during training. These findings motivate the hypothesis that useful weighting depends on the gap between the policy's current behavior and the behavior favored by the reward. (3) Guided by this analysis, we study simple static weighting, budgeted profile selection, and dynamic schedules that improve performance beyond conventional target choices. Our results establish timestep weighting as an important design choice for flow-matching RL and motivate further research into methods that choose and adapt it throughout learning.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2609.32665 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:29:02.430Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).