Metraipolis › archive › dc_8037bc7f425b
DiffusionRVGE: Reinforcement Learning via Regression-Based Value Gradient Estimation — arXiv:2610.22973
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.22973
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T08:42:21.214Z
- SHA-256 of the archived text
c2a556bf0252610cc28c6dfb4ed9935717fce9e06c84dec824d59989326679d9- Stable URL of this page
https://metraipolis.com/documento/dc_8037bc7f425b
How to cite
DiffusionRVGE: Reinforcement Learning via Regression-Based Value Gradient Estimation — arXiv:2610.22973 — arXiv (en), https://arxiv.org/abs/2610.22973 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8037bc7f425b (Metraipolis archive dc_8037bc7f425b, sha256 c2a556bf0252610cc28c6dfb4ed9935717fce9e06c84dec824d59989326679d9)
BibTeX
@misc{dc8037bc7f425b,
title = {DiffusionRVGE: Reinforcement Learning via Regression-Based Value Gradient Estimation — arXiv:2610.22973},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.22973},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8037bc7f425b; sha256 c2a556bf0252610cc28c6dfb4ed9935717fce9e06c84dec824d59989326679d9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.22973. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Reinforcement learning improves diffusion models using rewards on generated images, but repeated denoising and gradient estimation make training expensive. We introduce \method, an efficient method that constructs useful update directions by regressing centered rewards on centered final denoised images. It requires neither reward derivatives nor adjoint backpropagation through the sampling trajectory. A unified first-variation formulation identifies the value-gradient target with or without a trajectory-KL penalty. A population analysis under Gaussian noising explains the regression geometry, while a subspace analysis motivates clean-image coordinates. The implementation adapts rollout bandwidth and query weights to reward continuity: local rollouts suit smooth rewards, whereas broader groups provide informative contrasts for discrete rewards. Trained jointly on five rewards, one SD3.5-Medium model matches or exceeds DiffusionNFT's joint model on seven of eight metrics, reaching GenEval $0.94$, OCR $0.92$, and PickScore $23.82$ at about one fifth of its estimated training compute. In single-reward training, \method reaches matched reward thresholds $2.1$--$6.7\times$ faster than reproduced DiffusionNFT on identical hardware. Ablations support the clean-image estimator, direct direction transfer, and reward-dependent rollout choices. Code is available at this https URL . Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.22973 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T08:39:10.048Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).