Metraipolis › archive › dc_8fd141e4981e
Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Tasks — arXiv:2609.08404
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.08404
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T12:58:08.568Z
- SHA-256 of the archived text
549c5b25cfb57daf0bd1e6825e1b5d28e45e0c4aa2cb75baec0543163ba7d805- Stable URL of this page
https://metraipolis.com/documento/dc_8fd141e4981e
How to cite
Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Tasks — arXiv:2609.08404 — arXiv (en), https://arxiv.org/abs/2609.08404 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8fd141e4981e (Metraipolis archive dc_8fd141e4981e, sha256 549c5b25cfb57daf0bd1e6825e1b5d28e45e0c4aa2cb75baec0543163ba7d805)
BibTeX
@misc{dc8fd141e4981e,
title = {Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Tasks — arXiv:2609.08404},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.08404},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8fd141e4981e; sha256 549c5b25cfb57daf0bd1e6825e1b5d28e45e0c4aa2cb75baec0543163ba7d805},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.08404. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large Language Models demonstrate remarkable proficiency in static reasoning, yet training them as autonomous agents through Reinforcement Learning (RL) for long-horizon tasks is often hindered by severe reward sparsity. While conventional \textit{agent-side warming} up via supervised fine-tuning (SFT) can alleviate this, it is frequently limited by data scarcity and constrained exploration. To address this, we propose a paradigm shift to \textit{environment-side adaptation} by constructing \textbf{F}eedback-\textbf{E}nriched \textbf{E}nvironments (\textbf{FEEs}). Through a pilot study, we establish a feedback design strategy that reformulates environments by transitioning from action guidance to observation enrichment during the later stages of both intra-episode exploration and inter-episode evolution. Large-scale experiments on SciWorld and BFCL benchmarks using various Qwen3 model scales and RL algorithms such as GRPO, GSPO, and DAPO demonstrate that FEEs consistently yield performance improvements over standard settings. Furthermore, our analysis reveals that training with FEEs \textbf{(1)} stabilizes training dynamics by reducing entropy volatility, \textbf{(2)} facilitates proactive state-space exploration in difficult tasks, \textbf{(3) }ensures the internalization of environmental guidance into policy weights rather than acting as a mere inference-time prior, and \textbf{(4) }identifies intra-group feedback consistency as a critical boundary for stable optimization.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-09 in cs.CL).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.08404 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T12:30:25.355Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).