Metraipolis › archive › dc_dc7c4cfb77b7

Learning from Relative Action Credit for Escaping Local Minima — arXiv:2610.38034

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.38034
License (as archived)
CC0-1.0
Archived
2026-11-01T02:23:22.136Z
SHA-256 of the archived text
aa361fee52d718dd42971acb667dd3c1c81a15b9bea1541047e787c66ef8b3c4
Stable URL of this page
https://metraipolis.com/documento/dc_dc7c4cfb77b7

How to cite

Learning from Relative Action Credit for Escaping Local Minima — arXiv:2610.38034 — arXiv (en), https://arxiv.org/abs/2610.38034 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dc7c4cfb77b7 (Metraipolis archive dc_dc7c4cfb77b7, sha256 aa361fee52d718dd42971acb667dd3c1c81a15b9bea1541047e787c66ef8b3c4)
BibTeX
@misc{dcdc7c4cfb77b7,
  title = {Learning from Relative Action Credit for Escaping Local Minima — arXiv:2610.38034},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.38034},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dc7c4cfb77b7; sha256 aa361fee52d718dd42971acb667dd3c1c81a15b9bea1541047e787c66ef8b3c4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.38034. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Reinforcement learning holds the promise of robots that learn manipulation skills from interaction alone, without pretrained priors or demonstration data. In practice, on-policy methods such as PPO routinely stall on long-horizon, contact-rich tasks: dense reward shaping carries the policy through stages, yet stitching them together and discovering sparsely rewarded events rarely succeeds. We trace this failure to the missing action discrimination in state-value credit assignment: a state-value critic judges progress only by episode outcomes, so when the policy converges to a partial solution all outcomes coincide, and improvement stops because no action can prove better than another. We therefore extend PPO with an action conditioned value critic which, can compare the executed action with alternatives sampled at the same state and aid the learning signal. Combined with Generalized Advantage Estimation and a bound on exploration noise, we show how the extended algorithm escapes the plateau and completes precise peg insertion within a training budget where PPO with identical exploration fails.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2610.38034 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-01T02:12:44.098Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).