Metraipolis › archive › dc_f5fecc1fb698

Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments? — arXiv:2610.08215

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.08215
License (as archived)
CC0-1.0
Archived
2026-10-07T11:19:30.960Z
SHA-256 of the archived text
040935fe6f120aa59ce6fb266af2a56b6ae423eb911d714c2ee32b902225f419
Stable URL of this page
https://metraipolis.com/documento/dc_f5fecc1fb698

How to cite

Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments? — arXiv:2610.08215 — arXiv (en), https://arxiv.org/abs/2610.08215 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f5fecc1fb698 (Metraipolis archive dc_f5fecc1fb698, sha256 040935fe6f120aa59ce6fb266af2a56b6ae423eb911d714c2ee32b902225f419)
BibTeX
@misc{dcf5fecc1fb698,
  title = {Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments? — arXiv:2610.08215},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.08215},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f5fecc1fb698; sha256 040935fe6f120aa59ce6fb266af2a56b6ae423eb911d714c2ee32b902225f419},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.08215. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Learning from experience is essential for LLM agents to adapt to unfamiliar and dynmaic environments. Evaluating this ability is therefore important for understanding how effectively agents acquire and use new knowledge. Existing benchmarks have sought to evaluate this ability, but they primarily evaluate tasks whose rules are provided in the instructions or already familiar to pretrained models, making it difficult to distinguish learning from interactions from reasoning with existing knowledge. To address this, we introduce Learn2Play Bench, a benchmark of newly designed text-based games, whose rules are novel or counterintuitive, requiring agents to acquire knowledge through interaction rather than rely solely on pretrained knowledge. These games provide reproducible feedback and automatic scoring, enabling controlled evaluation of learning across repeated attempts. We also vary game instances to test whether agents can apply what they have learned to new situations. Therefore, we evaluate how backbone models, self-evolving methods, and agent harnesses affect agents' learning ability, revealing three findings: (1) Experience retention: Retaining complete records of actions and feedback can support more effective learning than summarizing these experiences into rules or strategies. (2) Human agent gap: Top-performing human players achieve higher peak scores than the evaluated agents. Human explore more varied strategies, and repeat actions less. (3) Harness matters: With the backbone fixed, changing the harness can improve performance while reducing estimated inference cost. Together, these findings provide insights into how LLM agents learn from experience and suggest directions for future work to improve their learning ability. Project website: this https URL

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.08215 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T11:15:24.594Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).