Metraipolis › archive › dc_69ebde02a7a9

GSBR: Geometric Support Behavior Regularization for Online RL — arXiv:2611.05295

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.05295
License (as archived)
CC0-1.0
Archived
2026-11-04T12:42:55.447Z
SHA-256 of the archived text
9d835db51e456368d666527989f51409b28e173f905063cd6463e48df924dbe9
Stable URL of this page
https://metraipolis.com/documento/dc_69ebde02a7a9

How to cite

GSBR: Geometric Support Behavior Regularization for Online RL — arXiv:2611.05295 — arXiv (en), https://arxiv.org/abs/2611.05295 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_69ebde02a7a9 (Metraipolis archive dc_69ebde02a7a9, sha256 9d835db51e456368d666527989f51409b28e173f905063cd6463e48df924dbe9)
BibTeX
@misc{dc69ebde02a7a9,
  title = {GSBR: Geometric Support Behavior Regularization for Online RL — arXiv:2611.05295},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.05295},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_69ebde02a7a9; sha256 9d835db51e456368d666527989f51409b28e173f905063cd6463e48df924dbe9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.05295. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Regularizing online reinforcement learning (RL) toward a pretrained base policy can improve sample efficiency and learning stability, but regularization that is too restrictive limits how much the policy can improve. Base policies -- typically diffusion or flow-matching models trained by imitation learning on large, diverse datasets -- capture many behaviors that are reasonable but suboptimal. Existing regularizers using such base policies are too restrictive in one of two ways: behavioral cloning regularizers pull the actor toward the full base-policy distribution, penalizing it for discarding suboptimal modes, while hard support constraints forbid improvements that lie just outside the base policy's support. We propose Geometric Support Behavior Regularization (GSBR), which penalizes actions by their distance from the base policy's support. Actions within the support incur no penalty, so the actor can concentrate on high-value modes; outside the support, where value estimates are less reliable, the penalty grows smoothly with distance while still permitting improvement when the value gained outweighs the penalty. GSBR estimates this distance from a finite set of base-policy samples, so it requires only sample access to the base policy. Across 19 tasks on four robotic benchmarks, GSBR raises the average success rate from 33% for the base policy to 95%, achieving the highest average success rate and throughput among all compared methods, with about 23% higher normalized throughput than the next-best RL method. Project website and open-source code at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Robotics (cs.RO) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.05295 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T12:29:47.413Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).