Metraipolis › archive › dc_2f6ef91f5df3

Dual-Channel Robust Group-Relative Policy Optimization via Advantage and Sequence-Weight Estimation — arXiv:2609.36944

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.36944
License (as archived)
CC0-1.0
Archived
2026-10-07T16:23:19.921Z
SHA-256 of the archived text
97d7b29715f3598c145d17017c4be482ce1433a5f48266030eeffd74f4592127
Stable URL of this page
https://metraipolis.com/documento/dc_2f6ef91f5df3

How to cite

Dual-Channel Robust Group-Relative Policy Optimization via Advantage and Sequence-Weight Estimation — arXiv:2609.36944 — arXiv (en), https://arxiv.org/abs/2609.36944 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2f6ef91f5df3 (Metraipolis archive dc_2f6ef91f5df3, sha256 97d7b29715f3598c145d17017c4be482ce1433a5f48266030eeffd74f4592127)
BibTeX
@misc{dc2f6ef91f5df3,
  title = {Dual-Channel Robust Group-Relative Policy Optimization via Advantage and Sequence-Weight Estimation — arXiv:2609.36944},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.36944},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2f6ef91f5df3; sha256 97d7b29715f3598c145d17017c4be482ce1433a5f48266030eeffd74f4592127},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.36944. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Group-relative policy optimization relies on reward-derived advantages and sequence-level likelihood weights, both of which can be sensitive to localized outliers. Extreme rewards can collapse the contrast among clean responses after group normalization, while token-level log-ratio perturbations can alter sequence weights and clipping decisions. We introduce RoVR-GSPO, a dual-channel robust optimizer that addresses these failure modes separately. Its reward channel combines robust reference estimation with bounded residual credit, while its ratio channel uses differentiable SoftRoVR aggregation to construct robust sequence weights. We provide stability and efficiency analyses for both channels. Experiments on mathematical reasoning, long-context summarization, and tool-call annotation show consistent improvements over GSPO, while controlled perturbation studies demonstrate stronger robustness to reward contamination and token-ratio anomalies.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.36944 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T15:54:53.571Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).