Metraipolis › archive › dc_fc428e328d83
OptionWorld: A Benchmark for Diagnosing Failures of LLMs as Long-Term Personal Assistants — arXiv:2610.28491
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.28491
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-21T07:09:10.646Z
- SHA-256 of the archived text
c71b1057e41b44eebfad598e5306f16a23da188327ccce962c2491d76c492680- Stable URL of this page
https://metraipolis.com/documento/dc_fc428e328d83
How to cite
OptionWorld: A Benchmark for Diagnosing Failures of LLMs as Long-Term Personal Assistants — arXiv:2610.28491 — arXiv (en), https://arxiv.org/abs/2610.28491 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fc428e328d83 (Metraipolis archive dc_fc428e328d83, sha256 c71b1057e41b44eebfad598e5306f16a23da188327ccce962c2491d76c492680)
BibTeX
@misc{dcfc428e328d83,
title = {OptionWorld: A Benchmark for Diagnosing Failures of LLMs as Long-Term Personal Assistants — arXiv:2610.28491},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.28491},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fc428e328d83; sha256 c71b1057e41b44eebfad598e5306f16a23da188327ccce962c2491d76c492680},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.28491. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Long-term assistants must follow unfinished tasks across days as information and circumstances change. Finding information or starting an action, however, does not ensure that the task gets completed. We introduce OptionWorld, a benchmark that traces information acquisition, judgments, actions, and outcomes in simulated daily-life tasks to identify where current LLMs fall short as long-term personal assistants. In runs with archived decision inputs, the key facts remain in the model's last input before the deadline in 35.5% of failed instances. Failure analysis identifies the three most frequent problems: needed information is not obtained or combined, available opportunities to act go unused, and agents impose restrictions that the task does not require. Three follow-up experiments show that supplying missing facts restores task success in some cases; action reminders can restart work while subsequent steps still miss the deadline; and correcting a supplied note that wrongly forbids contact restores permitted inquiry. These findings locate shortcomings in maintaining task state: connecting facts to unfinished work, following through before deadlines, and separating actual restrictions from the agent's assumptions. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.28491 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-21T07:05:58.156Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).