Metraipolis › archive › dc_a34cb35072ae

Reachability-Aware Diffusion Policy Optimization — arXiv:2610.05969

Source
arXiv
Original
https://arxiv.org/abs/2610.05969
License (as archived)
CC0-1.0
Archived
2026-10-06T05:31:23.259Z
SHA-256 of the archived text
64d5ef17f0be58907a7fbeb8b943d9cf607c488ab178582312921feaa2c8174c
Stable URL of this page
https://metraipolis.com/documento/dc_a34cb35072ae

How to cite

Reachability-Aware Diffusion Policy Optimization — arXiv:2610.05969 — arXiv, https://arxiv.org/abs/2610.05969 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a34cb35072ae (Metraipolis archive dc_a34cb35072ae, sha256 64d5ef17f0be58907a7fbeb8b943d9cf607c488ab178582312921feaa2c8174c)
BibTeX
@misc{dca34cb35072ae,
  title = {Reachability-Aware Diffusion Policy Optimization — arXiv:2610.05969},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.05969},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a34cb35072ae; sha256 64d5ef17f0be58907a7fbeb8b943d9cf607c488ab178582312921feaa2c8174c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.05969. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Diffusion policies provide expressive action distributions for continuous-control reinforcement learning. However, safety-aware online diffusion policy optimization remains underexplored, particularly methods that use predictive reachability information without an explicit dynamics model. We propose Reachability-Aware Diffusion Policy Optimization (RADPO), a model-free method that combines predictive first-hit safety estimation with cumulative-cost budget feedback. RADPO learns a discounted first-hit reachability value that captures the discounted risk of a cost event, assigns larger weight to events that occur sooner, and uses this signal to shape the reward. A separate dual-like multiplier adjusts the shaping strength according to realized episodic costs relative to a prescribed budget. The diffusion actor improves through weighted denoising regression on candidate actions scored by the reward critic. Our approach requires neither a learned dynamics model, action gradients through the critics, nor differentiation through the reverse diffusion sampler. We establish theoretical properties of the reachability value and show that accumulated reachability penalty provides a conservative surrogate for future discounted cumulative cost. Across ten continuous-control safety tasks, RADPO achieves competitive reward-cost trade-offs, with substantial reductions in constraint violations on several tasks relative to the compared baselines. Our theoretical and empirical analysis supports that combining reachability with cumulative budget feedback is a viable approach to safety-aware diffusion policies.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2610.05969 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T05:27:05.845Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).