Metraipolis › archive › dc_3dadc7827475

Learning from Viable Failure Prefixes: Milestone Viability Potential Policy Optimization for Long-Horizon LLM Agents — arXiv:2609.37111

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.37111
License (as archived)
CC0-1.0
Archived
2026-10-07T16:23:29.545Z
SHA-256 of the archived text
cd69a827d5fb6fb4ce1753fd336dd62537eb7b3434c60d6459b9c6f44af9104b
Stable URL of this page
https://metraipolis.com/documento/dc_3dadc7827475

How to cite

Learning from Viable Failure Prefixes: Milestone Viability Potential Policy Optimization for Long-Horizon LLM Agents — arXiv:2609.37111 — arXiv (en), https://arxiv.org/abs/2609.37111 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3dadc7827475 (Metraipolis archive dc_3dadc7827475, sha256 cd69a827d5fb6fb4ce1753fd336dd62537eb7b3434c60d6459b9c6f44af9104b)
BibTeX
@misc{dc3dadc7827475,
  title = {Learning from Viable Failure Prefixes: Milestone Viability Potential Policy Optimization for Long-Horizon LLM Agents — arXiv:2609.37111},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.37111},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3dadc7827475; sha256 cd69a827d5fb6fb4ce1753fd336dd62537eb7b3434c60d6459b9c6f44af9104b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.37111. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Long-horizon LLM agents require reinforcement learning methods that can assign credit to intermediate decisions under sparse and delayed rewards. Existing group-based methods such as GRPO and GiGPO alleviate this issue by comparing rollout returns or repeated anchor states, but they still fail when the compared returns have no variation. We identify this failure mode as zero-credit failure: during early training, many failed rollouts contain useful prefixes, yet existing methods assign them no task-discriminative advantage. To address this issue, we propose Milestone Viability Potential Policy Optimization (MVPO), a potential-routed policy optimization algorithm that learns from viable failure prefixes. MVPO estimates prefix potential over Union-Find viability regions, repairs zero-credit groups with potential-difference advantages, and attenuates the potential branch according to relative performance progress. Experiments with Qwen2.5-1.5B-Instruct show that MVPO outperforms eight strong baselines, including GRPO and GiGPO. Under the same training length, MVPO improves over the GiGPO baseline by +4.4 success points on ALFWorld and +5.3 on WebShop, while adding only 0.16%-0.20% advantage-construction overhead.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.37111 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T15:54:53.588Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).