Metraipolis › archive › dc_d4e026701904

Prism-GRPO: Faster VLA Policy Optimization via Splitting Same-outcome Groups — arXiv:2608.17423

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.17423
License (as archived)
CC0-1.0
Archived
2026-10-18T14:49:03.887Z
SHA-256 of the archived text
e0c44a30b7d59bc62c37a95ebe90ca3d1eac5f0391ef13663aa7afe8bd92c4e2
Stable URL of this page
https://metraipolis.com/documento/dc_d4e026701904

How to cite

Prism-GRPO: Faster VLA Policy Optimization via Splitting Same-outcome Groups — arXiv:2608.17423 — arXiv (en), https://arxiv.org/abs/2608.17423 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d4e026701904 (Metraipolis archive dc_d4e026701904, sha256 e0c44a30b7d59bc62c37a95ebe90ca3d1eac5f0391ef13663aa7afe8bd92c4e2)
BibTeX
@misc{dcd4e026701904,
  title = {Prism-GRPO: Faster VLA Policy Optimization via Splitting Same-outcome Groups — arXiv:2608.17423},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.17423},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d4e026701904; sha256 e0c44a30b7d59bc62c37a95ebe90ca3d1eac5f0391ef13663aa7afe8bd92c4e2},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.17423. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

GRPO is increasingly used for reinforcement learning of vision-language-action (VLA) policies because, unlike PPO, it does not require training a critic. This simplification comes with a sampling cost: group-relative advantages require multiple rollouts from each scene. Under binary success rewards, groups whose rollouts all succeed or all fail have zero advantage and are discarded by dynamic sampling. These groups are especially common early in training, when most rollouts fail, wasting much of the expensive robotic rollout budget. We introduce Prism-GRPO, which augments binary outcome reward with a weighted trajectory-level execution-quality score. By splitting same-outcome groups into a quality spectrum, Prism-GRPO recovers training signal while ensuring that every success still outranks every failure. Quality scores can be derived from simulator contacts, executed actions, or visual observations, avoiding task-specific progress rewards. We prove that Prism-GRPO never increases the probability that a sampled group is discarded for having zero advantages, and derive a gradient-alignment condition under which its combined update remains a local ascent direction for task success. Across four RoboTwin tasks spanning different horizons and coordination patterns, Prism-GRPO improves success and quality at matched rollout budgets and reaches target success rates with up to 56% fewer rollouts. It also suppresses a reward-hacking shortcut, with the cleaner behavior transferring under direct deployment to a real robot. Through ablations, we show consistent gains across contact-, smoothness-, and VLM-derived quality signals.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-19 in cs.LG).
Categorie arXiv: Robotics (cs.RO) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.17423 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T14:47:59.459Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-19?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).