Metraipolis › archive › dc_c2574a820bd2

AgenticFinBench: Benchmarking Financial Agents on Diverse Tasks from Real-World Requests — arXiv:2611.04813

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.04813
License (as archived)
CC0-1.0
Archived
2026-11-04T18:34:47.075Z
SHA-256 of the archived text
2eb783c62c33ab2f289b427f653aac92c9f060ab41fab09938d14dbd6887b291
Stable URL of this page
https://metraipolis.com/documento/dc_c2574a820bd2

How to cite

AgenticFinBench: Benchmarking Financial Agents on Diverse Tasks from Real-World Requests — arXiv:2611.04813 — arXiv (en), https://arxiv.org/abs/2611.04813 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c2574a820bd2 (Metraipolis archive dc_c2574a820bd2, sha256 2eb783c62c33ab2f289b427f653aac92c9f060ab41fab09938d14dbd6887b291)
BibTeX
@misc{dcc2574a820bd2,
  title = {AgenticFinBench: Benchmarking Financial Agents on Diverse Tasks from Real-World Requests — arXiv:2611.04813},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.04813},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c2574a820bd2; sha256 2eb783c62c33ab2f289b427f653aac92c9f060ab41fab09938d14dbd6887b291},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.04813. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Financial agents are becoming an important class of real-world AI systems. Financial tasks vary widely in type and difficulty, and many are open-ended with no ground-truth answers. However, existing benchmarks neither capture this breadth and open-endedness nor separate grounding from correctness. To bridge this gap, we introduce $\textbf{AgenticFinBench}$, a benchmark of 230 real-world financial tasks spanning 8 categories, 2 languages, and 4 complexity levels. We also propose a reference-free evaluation framework that combines question-specific rubrics with a two-stage claim audit that detects hallucinations, defined as claims absent from the agent's trace and refuted by external evidence. This framework better matches how financial work is assessed in practice. Experiments across 18 agent configurations reveal three key findings: (1) Ignoring hallucinations can overstate agent performance: hallucination auditing lowers the baseline leader's score by 7.58 points and moves it to fourth place. (2) Harness differences explain more variation in evidence grounding than model differences (28.6% vs 8.8%). (3) Quality scores fall from 89.44 on direct retrieval to 79.18 on the most complex tasks. Case studies show that agents struggle to explain how events affect company profits or stock prices, compare financial data on a consistent basis, and justify their conclusions. By releasing AgenticFinBench, we aim to advance financial reasoning, factual verification, and agent design for more reliable responses to diverse real-world financial requests. Our project page is at $\textbf{ this https URL }$.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.04813 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T18:32:37.338Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).