Metraipolis › archive › dc_2dbc4ca9bfb6

GSBR: Geometric Support Behavior Regularization for Online RL — arXiv:2611.05295

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.05295
License (as archived)
CC0-1.0
Archived
2026-11-04T18:56:07.677Z
SHA-256 of the archived text
08dfc04a925c598b09b7edfcb9cc56573f7e7decb1cb1a4bb1780c5a13d0919c
Stable URL of this page
https://metraipolis.com/documento/dc_2dbc4ca9bfb6

How to cite

GSBR: Geometric Support Behavior Regularization for Online RL — arXiv:2611.05295 — arXiv (en), https://arxiv.org/abs/2611.05295 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2dbc4ca9bfb6 (Metraipolis archive dc_2dbc4ca9bfb6, sha256 08dfc04a925c598b09b7edfcb9cc56573f7e7decb1cb1a4bb1780c5a13d0919c)
BibTeX
@misc{dc2dbc4ca9bfb6,
  title = {GSBR: Geometric Support Behavior Regularization for Online RL — arXiv:2611.05295},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.05295},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2dbc4ca9bfb6; sha256 08dfc04a925c598b09b7edfcb9cc56573f7e7decb1cb1a4bb1780c5a13d0919c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.05295. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Regularizing online reinforcement learning (RL) toward a pretrained base policy can improve sample efficiency and learning stability, but regularization that is too restrictive limits how much the policy can improve. Base policies -- typically diffusion or flow-matching models trained by imitation learning on large, diverse datasets -- capture many behaviors that are reasonable but suboptimal. Existing regularizers using such base policies are too restrictive in one of two ways: behavioral cloning regularizers pull the actor toward the full base-policy distribution, penalizing it for discarding suboptimal modes, while hard support constraints forbid improvements that lie just outside the base policy's support. We propose Geometric Support Behavior Regularization (GSBR), which penalizes actions by their distance from the base policy's support. Actions within the support incur no penalty, so the actor can concentrate on high-value modes; outside the support, where value estimates are less reliable, the penalty grows smoothly with distance while still permitting improvement when the value gained outweighs the penalty. GSBR estimates this distance from a finite set of base-policy samples, so it requires only sample access to the base policy. Across 19 tasks on four robotic benchmarks, GSBR raises the average success rate from 33% for the base policy to 95%, achieving the highest average success rate and throughput among all compared methods, with about 23% higher normalized throughput than the next-best RL method. Project website and open-source code at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Robotics (cs.RO) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.05295 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T18:35:37.555Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).