Metraipolis › archive › dc_dfc04943cb0e
VRL-Bench: Benchmarking agents on computer control tasks under finite trial budgets — arXiv:2609.12404
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.12404
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T03:02:44.239Z
- SHA-256 of the archived text
d65baded43fc0e22526996608a85977deca11859c350df8682cbdcf391add39b- Stable URL of this page
https://metraipolis.com/documento/dc_dfc04943cb0e
How to cite
VRL-Bench: Benchmarking agents on computer control tasks under finite trial budgets — arXiv:2609.12404 — arXiv (en), https://arxiv.org/abs/2609.12404 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dfc04943cb0e (Metraipolis archive dc_dfc04943cb0e, sha256 d65baded43fc0e22526996608a85977deca11859c350df8682cbdcf391add39b)
BibTeX
@misc{dcdfc04943cb0e,
title = {VRL-Bench: Benchmarking agents on computer control tasks under finite trial budgets — arXiv:2609.12404},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.12404},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dfc04943cb0e; sha256 d65baded43fc0e22526996608a85977deca11859c350df8682cbdcf391add39b},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.12404. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Learning from trial and error is a promising way to improve language agents on complex tasks such as computer control. Reflexion introduced verbal reinforcement learning, which turns failed trials into text that guides later attempts without updating model parameters. We introduce VRL-Bench, a harness for fair evaluation of trial-and-error learning under finite trial budgets. Across three models on MiniWoB and WebShop, we evaluate updates from several prominent verbal-memory methods spanning Reflexion and later work: each improves observed success over memory-free retry in some settings but reduces it in others. Replay experiments show that using reflection can reduce success rates, revealing a trade-off between exploiting experience and continued exploration. We propose VEX$^2$, a verbal exploration--exploitation scheduler that uses a language model to jointly select policies and allocate the remaining trial budget. VEX$^2$ is the only evaluated update to achieve positive observed success-rate gains over retry in all six settings. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-14 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.12404 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T02:51:27.789Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).