Metraipolis › archive › dc_d2bac978b591

Bank-World: A Benchmark for Evaluating Agents on Banking Workflows — arXiv:2610.39733

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.39733
License (as archived)
CC0-1.0
Archived
2026-10-31T06:09:25.757Z
SHA-256 of the archived text
a66dddd1a923b197bb55bed9b6e704383662f345415a393a7d8ca84a0c127525
Stable URL of this page
https://metraipolis.com/documento/dc_d2bac978b591

How to cite

Bank-World: A Benchmark for Evaluating Agents on Banking Workflows — arXiv:2610.39733 — arXiv (en), https://arxiv.org/abs/2610.39733 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d2bac978b591 (Metraipolis archive dc_d2bac978b591, sha256 a66dddd1a923b197bb55bed9b6e704383662f345415a393a7d8ca84a0c127525)
BibTeX
@misc{dcd2bac978b591,
  title = {Bank-World: A Benchmark for Evaluating Agents on Banking Workflows — arXiv:2610.39733},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.39733},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d2bac978b591; sha256 a66dddd1a923b197bb55bed9b6e704383662f345415a393a7d8ca84a0c127525},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.39733. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Bank-World is a benchmark of 50 banking tasks across two simulated institutions. Completing banking work requires more than producing the right calculation: agents must reconcile records, obtain permission, execute transactions, and keep instructions and status reports consistent as circumstances change. Bank-World tests these requirements through single-turn assignments and multi-turn interactions with simulated employees who supply information, approve actions, and revise instructions. Grading combines executable checks of recorded actions and state with model judgments of task requirements, including whether valid alternative solutions satisfy them. Across 1,600 attempts by four models, we find that correct decisions often fail to become completed work. Agents charge fees twice, distribute the correct total in incorrect amounts among the recipients, and execute payments while approval is pending. Their strengths also differ: Sol leads single-turn completion, Opus has the highest observed multi-turn completion rate, and Grok succeeds across more multi-turn tasks but less consistently. Bank-World makes these differences inspectable through task-level results and worked examples connecting source evidence, agent decisions, and simulated consequences. We retain the task environments, executable graders, and full trajectories privately to reduce contamination and preserve the benchmark for future evaluations.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.39733 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T06:07:08.536Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).