Metraipolis › archive › dc_4c8bfa4ca82a
SCQ: Stabilizing Conservative Q-Learning with Sigmoid-Bounded Entropy — arXiv:2609.12749
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.12749
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T03:02:54.830Z
- SHA-256 of the archived text
71cc8cc391922e37dd732274bf509ca8d8fe4911d103cfefc75567ee0d63f4c0- Stable URL of this page
https://metraipolis.com/documento/dc_4c8bfa4ca82a
How to cite
SCQ: Stabilizing Conservative Q-Learning with Sigmoid-Bounded Entropy — arXiv:2609.12749 — arXiv (en), https://arxiv.org/abs/2609.12749 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4c8bfa4ca82a (Metraipolis archive dc_4c8bfa4ca82a, sha256 71cc8cc391922e37dd732274bf509ca8d8fe4911d103cfefc75567ee0d63f4c0)
BibTeX
@misc{dc4c8bfa4ca82a,
title = {SCQ: Stabilizing Conservative Q-Learning with Sigmoid-Bounded Entropy — arXiv:2609.12749},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.12749},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4c8bfa4ca82a; sha256 71cc8cc391922e37dd732274bf509ca8d8fe4911d103cfefc75567ee0d63f4c0},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.12749. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Offline-to-online reinforcement learning reduces interaction cost for real-world robot learning but suffers from persistent value estimation instability. Existing methods address this through pessimistic regularization, lower-bound calibration, and architectural normalization, but an overlooked source of instability lies in the entropy formulation: the standard log-entropy term can become negative, destabilizing policy updates. We introduce SCQ (Sigmoid-Bounded Conservative Q-Learning), which replaces this term with a sigmoid-bounded formulation that stays strictly positive. SCQ retains conservative Q regularization and return-based lower-bound calibration, stabilizing policy optimization without sacrificing exploration. We evaluate SCQ on D4RL (Minari) benchmarks under both single-demonstration and standard dataset settings, as well as on simulation and real-world visual tasks. SCQ matches or exceeds baseline performance while exhibiting more stable training dynamics across state-based and visual benchmarks, and transfers to four real-robot platforms including manipulation, wheeled, quadruped, and humanoid systems. A direct clipping intervention that removes negative log-probability contributions, together with gradient-matched positive-score controls, indicates that positivity rather than a particular score shape alone drives much of the improvement. Project website: this https URL . Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-14 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.12749 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T02:51:27.819Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).