Metraipolis › archive › dc_6bfd7693d383
AutoBuddy: Evaluating Proactive, Always-On Agents on Task Discovery and Execution in a Simulated Workplace — arXiv:2611.12393
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.12393
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-10T18:41:53.392Z
- SHA-256 of the archived text
b7f8e81135be874f03db74cdc4b29f82b1dbb0ff2665bc1814801979cad22f30- Stable URL of this page
https://metraipolis.com/documento/dc_6bfd7693d383
How to cite
AutoBuddy: Evaluating Proactive, Always-On Agents on Task Discovery and Execution in a Simulated Workplace — arXiv:2611.12393 — arXiv (en), https://arxiv.org/abs/2611.12393 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6bfd7693d383 (Metraipolis archive dc_6bfd7693d383, sha256 b7f8e81135be874f03db74cdc4b29f82b1dbb0ff2665bc1814801979cad22f30)
BibTeX
@misc{dc6bfd7693d383,
title = {AutoBuddy: Evaluating Proactive, Always-On Agents on Task Discovery and Execution in a Simulated Workplace — arXiv:2611.12393},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.12393},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6bfd7693d383; sha256 b7f8e81135be874f03db74cdc4b29f82b1dbb0ff2665bc1814801979cad22f30},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.12393. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
As LLM agents become more autonomous, they are increasingly expected to run as always-on assistants on a user's behalf, identifying work to be done and completing it without being asked. Yet existing benchmarks usually give the agent its task, whereas in knowledge work the requests sit in the user's workspace and the agent must first recognise which obligations exist. We study proactive task discovery and execution, in which an agent must infer a user's pending tasks from their workspace and complete them in the systems where the work lives. This setting poses two challenges. For the agent, the tasks are buried in a large volume of mail, chat, documents and records. For evaluation, a benchmark needs a simulated organisation whose people, events and documents are mutually consistent, so that tasks arise realistically and their correct outcomes are known. We introduce AutoBuddy-Bench, a simulator that addresses these challenges. It models an organisation's employees, projects, business systems and event history as a temporal graph, renders it into realistic workspace artefacts, derives ground-truth tasks from the events that create obligations, and computes each task's outcome from the world state; execution is judged by the agent's changes to the workspace. To evaluate current LLMs, we build AutoBuddy, a reference agent with interchangeable backbone LLMs: it identifies tasks from the workspace, grounding each in its source records, and executes each in a separate session with write access. Varying only the backbone, we find a systematic shift in where tasks are lost: weaker models fail mainly at discovery, intermediate models identify tasks but often stop short of carrying them out, and the strongest models fail mainly during execution, most often on business-system operations. Discovery precision is only weakly related to execution quality, suggesting the two are distinct capabilities. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2611.12393 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-10T18:37:37.551Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).