Metraipolis › archive › dc_f44b413708d1

Safe Score Matching: Diffusion Policies with Hamilton-Jacobi Reachability for Online Safe Reinforcement Learning — arXiv:2609.33337

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.33337
License (as archived)
CC0-1.0
Archived
2026-10-07T22:37:23.395Z
SHA-256 of the archived text
5806e2ce4596e021456e572a91bacf864af16c56f04be3ecc9a25325d4e41ac7
Stable URL of this page
https://metraipolis.com/documento/dc_f44b413708d1

How to cite

Safe Score Matching: Diffusion Policies with Hamilton-Jacobi Reachability for Online Safe Reinforcement Learning — arXiv:2609.33337 — arXiv (en), https://arxiv.org/abs/2609.33337 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f44b413708d1 (Metraipolis archive dc_f44b413708d1, sha256 5806e2ce4596e021456e572a91bacf864af16c56f04be3ecc9a25325d4e41ac7)
BibTeX
@misc{dcf44b413708d1,
  title = {Safe Score Matching: Diffusion Policies with Hamilton-Jacobi Reachability for Online Safe Reinforcement Learning — arXiv:2609.33337},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.33337},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f44b413708d1; sha256 5806e2ce4596e021456e572a91bacf864af16c56f04be3ecc9a25325d4e41ac7},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33337. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Online safe reinforcement learning (RL) seeks policies that maximize reward while satisfying safety constraints. A popular line of research in safe RL relaxes safety to a soft expected-cost constraint and solves the resulting Constrained Markov Decision Process via primal-dual Lagrangian updates that only enforce safety on average. To address this limitation, hard, state-wise constraints are introduced and often imposed through Hamilton-Jacobi (HJ) reachability. Yet such constraints require solving different objectives in the feasible and infeasible regions: reward maximization in the former, recovery toward the feasible regions in the latter. The resulting target action distributions are inherently multimodal, and this structure poses a fundamental challenge for the Gaussian or deterministic actors used in existing HJ-based safe RL, which often collapse onto suboptimal modes. Diffusion policies provide the expressiveness needed to represent such distributions, and recent work on Q-score matching offers a route to training them for online RL by score regression -- but has been applied only to reward maximization. We propose Safe Score Matching (SSM), an off-policy actor-critic method that adapts Q-score matching to hard-constrained safe RL by gating a two-branch score target with HJ reachability: inside the feasible set, the denoising process degenerates to Q-score matching on actions classified as viable by the HJ critic; outside, a recovery branch biases denoising toward regions with lower worst-case violation. On quadrotor and fixed-wing trajectory-tracking and stabilize-and-avoid benchmarks, SSM attains the best or near-best task performance with low false-safe rates, whereas the primal-dual baseline admits more unsafe behavior and reachability-based baselines tend to be more conservative; on Safety-Gymnasium velocity tasks, SSM attains the lowest cost with competitive reward.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2609.33337 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T22:14:27.920Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).