Metraipolis › archive › dc_f5db58bfd122

DAYJOB: A Benchmark for Long-Horizon Professional Work — arXiv:2610.01306

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.01306
License (as archived)
CC0-1.0
Archived
2026-10-06T12:04:35.678Z
SHA-256 of the archived text
957eec659099f6aa867feb4f4d808ad6c2357d4e6a487cce61d25ed6e7534fe4
Stable URL of this page
https://metraipolis.com/documento/dc_f5db58bfd122

How to cite

DAYJOB: A Benchmark for Long-Horizon Professional Work — arXiv:2610.01306 — arXiv (en), https://arxiv.org/abs/2610.01306 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f5db58bfd122 (Metraipolis archive dc_f5db58bfd122, sha256 957eec659099f6aa867feb4f4d808ad6c2357d4e6a487cce61d25ed6e7534fe4)
BibTeX
@misc{dcf5db58bfd122,
  title = {DAYJOB: A Benchmark for Long-Horizon Professional Work — arXiv:2610.01306},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.01306},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f5db58bfd122; sha256 957eec659099f6aa867feb4f4d808ad6c2357d4e6a487cce61d25ed6e7534fe4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.01306. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Professional work often starts with a brief request that leaves the professional to work out what is needed, which documents matter, and whether the request's premise holds. We introduce DAYJOB, a benchmark of 130 tasks built by professionals in healthcare (50) and finance (80). The tasks are estimated to take a professional 13.6 hours on average in healthcare and 16.6 in finance. Each task is a containerized Harbor environment with an expert rubric of binary criteria (median 47.5 and 57.5 per task) that an agentic judge applies to the delivered files, and an attempt passes only if it meets every criterion. Across 30 model configurations from 13 developers, the strongest, Claude Opus 5.5, passes 24.7% of healthcare and 23.9% of finance attempts, and the median configuration passes 0.6% and 2.5%. In case studies, agents accept premises that the record contradicts and carry wrong inputs through otherwise consistent analyses. We release all healthcare tasks, 50 of the 80 finance tasks, the evaluation harness, and the leaderboard.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-02 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.01306 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T10:21:27.936Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).