Metraipolis › archive › dc_1b8934d284fa
Explorer-Zero: Improving Self-Play for Long-Horizon Tasks with Decomposable Verifiable Rewards — arXiv:2610.19868
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.19868
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T14:23:39.657Z
- SHA-256 of the archived text
ecc6521c71b7d4255178aa07a8c065951a58757cb4aa700549cfe11baacc938d- Stable URL of this page
https://metraipolis.com/documento/dc_1b8934d284fa
How to cite
Explorer-Zero: Improving Self-Play for Long-Horizon Tasks with Decomposable Verifiable Rewards — arXiv:2610.19868 — arXiv (en), https://arxiv.org/abs/2610.19868 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_1b8934d284fa (Metraipolis archive dc_1b8934d284fa, sha256 ecc6521c71b7d4255178aa07a8c065951a58757cb4aa700549cfe11baacc938d)
BibTeX
@misc{dc1b8934d284fa,
title = {Explorer-Zero: Improving Self-Play for Long-Horizon Tasks with Decomposable Verifiable Rewards — arXiv:2610.19868},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.19868},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_1b8934d284fa; sha256 ecc6521c71b7d4255178aa07a8c065951a58757cb4aa700549cfe11baacc938d},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.19868. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Training LLM agents on long-horizon interactive tasks with reinforcement learning typically relies on human-annotated tasks and reliable verification, both expensive to construct for stateful environments. Self-play offers an alternative, but must generate achievable, compositional tasks and provide feedback beyond binary outcomes. We propose Explorer-Zero, a self-play framework that jointly generates tasks and fine-grained, verifiable rewards through environment interaction. A learned proposer explores the environment, constructs tasks, and records intermediate and final states as verification targets. These checkpoints reward partial completion, while alternating proposer--solver training adapts the generated curriculum. We show theoretically that subtask rewards reduce collapse among unsuccessful rollouts when they complete different numbers of subtasks. Without human-annotated training tasks, Explorer-Zero outperforms the evaluated self-play baselines across ALFWorld, ScienceWorld, and AppWorld; on AppWorld, our 30B agent reaches 59.67% task success, approaching 62.65% for RL on human-annotated tasks. Further analyses support task-relevant composition and benefits on complex tasks, while self-play initialization improves subsequent human-data RL across all three benchmarks. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.19868 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T13:53:35.918Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).