Metraipolis › archive › dc_101ae05ed136
JarvisBench: Always-on Intelligence Between Humans and Agents — arXiv:2608.14870
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.14870
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T17:08:12.575Z
- SHA-256 of the archived text
bd682dbfb465f8d03d6b29c9a0c1d768c0394606a20f77be80fa235677108b16- Stable URL of this page
https://metraipolis.com/documento/dc_101ae05ed136
How to cite
JarvisBench: Always-on Intelligence Between Humans and Agents — arXiv:2608.14870 — arXiv (en), https://arxiv.org/abs/2608.14870 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_101ae05ed136 (Metraipolis archive dc_101ae05ed136, sha256 bd682dbfb465f8d03d6b29c9a0c1d768c0394606a20f77be80fa235677108b16)
BibTeX
@misc{dc101ae05ed136,
title = {JarvisBench: Always-on Intelligence Between Humans and Agents — arXiv:2608.14870},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.14870},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_101ae05ed136; sha256 bd682dbfb465f8d03d6b29c9a0c1d768c0394606a20f77be80fa235677108b16},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.14870. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Long-horizon agents can execute continuously, but human attention remains intermittent and scarce. This creates a bidirectional coordination problem: users may need immediate access to an agent while work continues in the background, whereas agents may encounter consequential decisions that require user judgment after the user has stopped monitoring execution. We posit an always-on attention-coordination layer---\textit{Jarvis}\footnote{Named after the fictional AI assistant in \textit{Iron Man}.}---that mediates this interface and allocates human attention across one or more working agents. We introduce \textit{JarvisBench} to evaluate both directions of this coordination: whether an intermediary can accurately and promptly answer user-initiated questions about ongoing work, and whether it can recognize when an agent requires user judgment, solicit that judgment at the right moment, and route it back to improve task outcomes. JarvisBench contains 45 agentic task instances: 20 single-agent tasks and 25 workstreams organized into 10 multi-agent projects. The tasks span 19 domains and were selected and adapted from more than 2,000 public candidates. Crucially, the need for user attention arises naturally during execution rather than from an obvious omission in the initial prompt. JarvisBench is designed to integrate with arbitrary agent runtimes without modifying their underlying execution loops. Our reference implementation further provides a full-duplex speech interface, allowing users to reach Jarvis naturally while timely attention coordination supports agents working in the background. By separating agent execution from attention coordination, JarvisBench provides a stable evaluation target as agent capabilities continue to improve.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.14870 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T16:29:13.743Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).