Metraipolis › archive › dc_ee2384df7f1a

FinAutoRubric: Expert-Guided Automatic Rubric Generation for Evaluating Financial Research Agents — arXiv:2609.35744

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.35744
License (as archived): CC0-1.0
Archived: 2026-10-07T20:59:05.216Z
SHA-256 of the archived text: ea2e9b9c600bb8d98f93ad7f8c7177e6770fed23f55802de38b2e449cd539661
Stable URL of this page: https://metraipolis.com/documento/dc_ee2384df7f1a

How to cite

FinAutoRubric: Expert-Guided Automatic Rubric Generation for Evaluating Financial Research Agents — arXiv:2609.35744 — arXiv (en), https://arxiv.org/abs/2609.35744 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ee2384df7f1a (Metraipolis archive dc_ee2384df7f1a, sha256 ea2e9b9c600bb8d98f93ad7f8c7177e6770fed23f55802de38b2e449cd539661)

BibTeX

@misc{dcee2384df7f1a,
  title = {FinAutoRubric: Expert-Guided Automatic Rubric Generation for Evaluating Financial Research Agents — arXiv:2609.35744},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.35744},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ee2384df7f1a; sha256 ea2e9b9c600bb8d98f93ad7f8c7177e6770fed23f55802de38b2e449cd539661},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.35744. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Evaluating finance research agents requires rubrics that reflect expert standards and fix the values correct as of an information cutoff. Expert-reviewed finance benchmarks rely on fixed, per-item rubrics, which are costly to extend and cannot encode each institution's own standard. In FinAutoRubric, experts specify reusable evaluation guidance, while agents and code carry out query-specific rubric generation, review, and validation. This expert guidance governs every agent, as prompts and as rules that code enforces, and a Task Bank of reusable criteria carries it across tasks. In long-horizon loops that follow the expert guidance, a writer agent researches every expected value and a reviewer agent verifies it, and failures escalate to a human. On three expert-authored finance benchmarks, its rubrics track expert scoring as closely as the strongest evaluated generator while stating the expert rubric's expected value for more criteria, their scores agree with human grading, and in-house analysts prefer them in a blind review. The released 100-query FinAutoRubric Benchmark, built from in-house analysts' key questions across 78 tasks and eight asset classes, shows that rubrics from an earlier model generation still leave headroom for a later one.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computational Finance (q-fin.CP).
Pagina del lavoro: https://arxiv.org/abs/2609.35744 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:29:01.942Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).