Metraipolis › archive › dc_6ea2589857dd
Noisy-Space Policy Gradient for Diffusion Policies in Offline Reinforcement Learning — arXiv:2609.06882
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.06882
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T13:57:36.408Z
- SHA-256 of the archived text
eb7e7477b55056f650b5967a4b9316cab56a1ee6064854826e7e8bf978904215- Stable URL of this page
https://metraipolis.com/documento/dc_6ea2589857dd
How to cite
Noisy-Space Policy Gradient for Diffusion Policies in Offline Reinforcement Learning — arXiv:2609.06882 — arXiv (en), https://arxiv.org/abs/2609.06882 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6ea2589857dd (Metraipolis archive dc_6ea2589857dd, sha256 eb7e7477b55056f650b5967a4b9316cab56a1ee6064854826e7e8bf978904215)
BibTeX
@misc{dc6ea2589857dd,
title = {Noisy-Space Policy Gradient for Diffusion Policies in Offline Reinforcement Learning — arXiv:2609.06882},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.06882},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6ea2589857dd; sha256 eb7e7477b55056f650b5967a4b9316cab56a1ee6064854826e7e8bf978904215},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.06882. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Diffusion policies offer a powerful and expressive parameterization for continuous control. Yet, their integration with reinforcement learning remains conceptually and algorithmically challenging. In this work, we address this gap by introducing a noisy-space action-value (Q-)function that assigns values to diffusion latents through the distribution of executed actions induced by the denoising process. We show that this construction admits a precise semantic interpretation and derive a noisy-space policy gradient (NSPG) that optimizes noisy latents using only clean action-space value estimates. Building on this result, we formulate a KL-regularized policy improvement over noisy latents and show that the resulting objective admits a diffusion-compatible regression form, avoiding backpropagation through the denoising process. Empirical results on state-based D4RL benchmarks and vision-based OGBench tasks demonstrate that the proposed noisy-space objective provides a principled and effective basis for training diffusion policies in offline reinforcement learning. Project webpage: this https URL Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-09 in cs.LG). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Robotics (cs.RO). Pagina del lavoro: https://arxiv.org/abs/2609.06882 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T13:39:42.915Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).