Metraipolis › archive › dc_1b8934d284fa

Explorer-Zero: Improving Self-Play for Long-Horizon Tasks with Decomposable Verifiable Rewards — arXiv:2610.19868

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.19868
License (as archived)
CC0-1.0
Archived
2026-10-15T14:23:39.657Z
SHA-256 of the archived text
ecc6521c71b7d4255178aa07a8c065951a58757cb4aa700549cfe11baacc938d
Stable URL of this page
https://metraipolis.com/documento/dc_1b8934d284fa

How to cite

Explorer-Zero: Improving Self-Play for Long-Horizon Tasks with Decomposable Verifiable Rewards — arXiv:2610.19868 — arXiv (en), https://arxiv.org/abs/2610.19868 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_1b8934d284fa (Metraipolis archive dc_1b8934d284fa, sha256 ecc6521c71b7d4255178aa07a8c065951a58757cb4aa700549cfe11baacc938d)
BibTeX
@misc{dc1b8934d284fa,
  title = {Explorer-Zero: Improving Self-Play for Long-Horizon Tasks with Decomposable Verifiable Rewards — arXiv:2610.19868},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.19868},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_1b8934d284fa; sha256 ecc6521c71b7d4255178aa07a8c065951a58757cb4aa700549cfe11baacc938d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.19868. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Training LLM agents on long-horizon interactive tasks with reinforcement learning typically relies on human-annotated tasks and reliable verification, both expensive to construct for stateful environments. Self-play offers an alternative, but must generate achievable, compositional tasks and provide feedback beyond binary outcomes. We propose Explorer-Zero, a self-play framework that jointly generates tasks and fine-grained, verifiable rewards through environment interaction. A learned proposer explores the environment, constructs tasks, and records intermediate and final states as verification targets. These checkpoints reward partial completion, while alternating proposer--solver training adapts the generated curriculum. We show theoretically that subtask rewards reduce collapse among unsuccessful rollouts when they complete different numbers of subtasks. Without human-annotated training tasks, Explorer-Zero outperforms the evaluated self-play baselines across ALFWorld, ScienceWorld, and AppWorld; on AppWorld, our 30B agent reaches 59.67% task success, approaching 62.65% for RL on human-annotated tasks. Further analyses support task-relevant composition and benefits on complex tasks, while self-play initialization improves subsequent human-data RL across all three benchmarks.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.19868 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T13:53:35.918Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).