Metraipolis › archive › dc_6daf010c016a

Design the Feedback, Not Just the Reward: Environments for Open-Ended Reinforcement Learning — arXiv:2608.27906

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.27906
License (as archived)
CC0-1.0
Archived
2026-10-17T10:47:17.877Z
SHA-256 of the archived text
f228b7ea4c5ef13c6b3994951bd8f265d513b29af18d47c0b8c8c045b64e52cc
Stable URL of this page
https://metraipolis.com/documento/dc_6daf010c016a

How to cite

Design the Feedback, Not Just the Reward: Environments for Open-Ended Reinforcement Learning — arXiv:2608.27906 — arXiv (en), https://arxiv.org/abs/2608.27906 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6daf010c016a (Metraipolis archive dc_6daf010c016a, sha256 f228b7ea4c5ef13c6b3994951bd8f265d513b29af18d47c0b8c8c045b64e52cc)
BibTeX
@misc{dc6daf010c016a,
  title = {Design the Feedback, Not Just the Reward: Environments for Open-Ended Reinforcement Learning — arXiv:2608.27906},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.27906},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6daf010c016a; sha256 f228b7ea4c5ef13c6b3994951bd8f265d513b29af18d47c0b8c8c045b64e52cc},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.27906. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Reinforcement learning has advanced large language models on tasks with reliable verifiers, where checking an outcome provides a clear basis for reward. Extending this approach to open-ended tasks requires environments that can both evaluate complex, partially successful outputs and turn these judgments into effective learning guidance. We propose a reinforcement learning framework built around a training environment that integrates human domain knowledge with model reasoning. Human domain knowledge, encoded in rubrics, anchors evaluation to task intent, while model reasoning combined with hierarchical rewards distinguishes complex open-ended outputs and links diagnosed failures to likely responsible output regions. Furthermore, these diagnostic attributions guide token-level weights in the GRPO objective, thereby translating the ambiguous notion of success in open-ended tasks into concrete guidance for policy optimization. We instantiate this framework in interactive web application generation, a representative open-ended generation task. The resulting model achieves an average pass rate of 41.25\% on MiniAppBench and 76.19\% on ArtifactsBench. Ablation studies show the contributions of individual components, while additional out-of-distribution evaluations support transferable gains without degrading performance on other evaluated tasks.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-31 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.27906 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T10:33:42.365Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-31?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).