Metraipolis › archive › dc_f44b413708d1
Safe Score Matching: Diffusion Policies with Hamilton-Jacobi Reachability for Online Safe Reinforcement Learning — arXiv:2609.33337
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.33337
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T22:37:23.395Z
- SHA-256 of the archived text
5806e2ce4596e021456e572a91bacf864af16c56f04be3ecc9a25325d4e41ac7- Stable URL of this page
https://metraipolis.com/documento/dc_f44b413708d1
How to cite
Safe Score Matching: Diffusion Policies with Hamilton-Jacobi Reachability for Online Safe Reinforcement Learning — arXiv:2609.33337 — arXiv (en), https://arxiv.org/abs/2609.33337 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f44b413708d1 (Metraipolis archive dc_f44b413708d1, sha256 5806e2ce4596e021456e572a91bacf864af16c56f04be3ecc9a25325d4e41ac7)
BibTeX
@misc{dcf44b413708d1,
title = {Safe Score Matching: Diffusion Policies with Hamilton-Jacobi Reachability for Online Safe Reinforcement Learning — arXiv:2609.33337},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.33337},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f44b413708d1; sha256 5806e2ce4596e021456e572a91bacf864af16c56f04be3ecc9a25325d4e41ac7},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33337. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Online safe reinforcement learning (RL) seeks policies that maximize reward while satisfying safety constraints. A popular line of research in safe RL relaxes safety to a soft expected-cost constraint and solves the resulting Constrained Markov Decision Process via primal-dual Lagrangian updates that only enforce safety on average. To address this limitation, hard, state-wise constraints are introduced and often imposed through Hamilton-Jacobi (HJ) reachability. Yet such constraints require solving different objectives in the feasible and infeasible regions: reward maximization in the former, recovery toward the feasible regions in the latter. The resulting target action distributions are inherently multimodal, and this structure poses a fundamental challenge for the Gaussian or deterministic actors used in existing HJ-based safe RL, which often collapse onto suboptimal modes. Diffusion policies provide the expressiveness needed to represent such distributions, and recent work on Q-score matching offers a route to training them for online RL by score regression -- but has been applied only to reward maximization. We propose Safe Score Matching (SSM), an off-policy actor-critic method that adapts Q-score matching to hard-constrained safe RL by gating a two-branch score target with HJ reachability: inside the feasible set, the denoising process degenerates to Q-score matching on actions classified as viable by the HJ critic; outside, a recovery branch biases denoising toward regions with lower worst-case violation. On quadrotor and fixed-wing trajectory-tracking and stabilize-and-avoid benchmarks, SSM attains the best or near-best task performance with low false-safe rates, whereas the primal-dual baseline admits more unsafe behavior and reachability-based baselines tend to be more conservative; on Safety-Gymnasium velocity tasks, SSM attains the lowest cost with competitive reward. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG). Categorie arXiv: Machine Learning (cs.LG) ; Robotics (cs.RO). Pagina del lavoro: https://arxiv.org/abs/2609.33337 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T22:14:27.920Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).