Metraipolis › archive › dc_403ffb1cfee2

SRPO: Self-Reflective Policy Optimization for Long-Horizon Reasoning — arXiv:2608.23493

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.23493
License (as archived): CC0-1.0
Archived: 2026-10-18T07:41:59.020Z
SHA-256 of the archived text: 074599533b5f7ab7a83af6359a0edcfbf245c62685dce3ed278bee1e751b86f7
Stable URL of this page: https://metraipolis.com/documento/dc_403ffb1cfee2

How to cite

SRPO: Self-Reflective Policy Optimization for Long-Horizon Reasoning — arXiv:2608.23493 — arXiv (en), https://arxiv.org/abs/2608.23493 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_403ffb1cfee2 (Metraipolis archive dc_403ffb1cfee2, sha256 074599533b5f7ab7a83af6359a0edcfbf245c62685dce3ed278bee1e751b86f7)

BibTeX

@misc{dc403ffb1cfee2,
  title = {SRPO: Self-Reflective Policy Optimization for Long-Horizon Reasoning — arXiv:2608.23493},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.23493},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_403ffb1cfee2; sha256 074599533b5f7ab7a83af6359a0edcfbf245c62685dce3ed278bee1e751b86f7},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.23493. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Self-reflection is a powerful mechanism for credit assignment in human learning, converting sparse outcome feedback into actionable guidance. However, its potential for post-training Large Language Models (LLMs) remains underexplored. We propose Self-Reflective Policy Optimization (SRPO), a framework that internalizes this capability. SRPO enables LLMs to analyze their own completed trajectories, synthesize errors into concise "reflection patches," and use reflection-conditioned teacher scores on student on-policy rollouts as dense token-level training signals. This process effectively transforms sparse terminal supervision into dense, token-level learning signals without requiring external critics, separate reward models, or larger teacher models. We demonstrate that SRPO achieves state-of-the-art performance across mathematical reasoning and long-horizon agentic benchmarks with exceptional data efficiency. Using a Qwen3-8B base model, SRPO attains 73.3% on AIME'24 using only 8% (0.08x) of the training FLOPs required by scaled supervised fine-tuning, while significantly improving success rates on WebShop (64.7%), ALFWorld (76.8%), and SWE-Bench-Lite (31.2%). Code is available at this https URL

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-25 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.23493 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T07:37:52.782Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).