Metraipolis › archive › dc_817323fca9ce
MBABench-2.0: Evaluating AI Agents on Constructing Financial Spreadsheets Beyond Simple Verifiability — arXiv:2610.31711
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.31711
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-26T19:50:54.624Z
- SHA-256 of the archived text
71451698d2ce65b8d2f2c2c51a0458646878cb554323082fc2a134c53b7fe20c- Stable URL of this page
https://metraipolis.com/documento/dc_817323fca9ce
How to cite
MBABench-2.0: Evaluating AI Agents on Constructing Financial Spreadsheets Beyond Simple Verifiability — arXiv:2610.31711 — arXiv (en), https://arxiv.org/abs/2610.31711 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_817323fca9ce (Metraipolis archive dc_817323fca9ce, sha256 71451698d2ce65b8d2f2c2c51a0458646878cb554323082fc2a134c53b7fe20c)
BibTeX
@misc{dc817323fca9ce,
title = {MBABench-2.0: Evaluating AI Agents on Constructing Financial Spreadsheets Beyond Simple Verifiability — arXiv:2610.31711},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.31711},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_817323fca9ce; sha256 71451698d2ce65b8d2f2c2c51a0458646878cb554323082fc2a134c53b7fe20c},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.31711. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Spreadsheets are the lingua franca of modern finance, driving valuation, forecast, audit, and many other core workflows of the industry. As LLM agents advance to automate knowledge work, spreadsheet construction is a natural and pertinent testbed. However, most existing spreadsheet benchmarks evaluate agents on fill-in-the-blank style tasks, where an agent performs minor edits in otherwise complete spreadsheets. This protocol fails to capture the most central and time-consuming part of the spreadsheet workflow: the modeling decisions, format design, and sheet layout regular knowledge workers need to toil through to build a workbook from scratch. We introduce MBABench-2.0, a benchmark of 101 original spreadsheet construction tasks, the hardest of which is estimated to take 33 hours for a modeler with 3+ years of experience. Unlike coding or math, where a unit test or a final answer can verify an agent's output, the quality of a financial spreadsheet that concerns readability, maintainability, or auditability, is not so simply verifiable. To holistically evaluate spreadsheet construction, we introduce 129 criteria spanning 12 categories that operationalize quality beyond the accuracy of the spreadsheet. Evaluating over 20 agents across 8 models and 4 harness types, we find that frontier agents rarely construct spreadsheets that are both accurate and meet our quality criteria: the top agent achieves a pass rate of only 16% on the benchmark. The benchmark demonstrates a substantial gap in spreadsheet building capability of current agents. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Software Engineering (cs.SE) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.31711 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-26T19:45:33.351Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).