Metraipolis › archive › dc_46ebf63d25bc

Better Deck or Different Judge? Evaluating Agentic Harness Gains in Corporate and Investment Banking — arXiv:2609.39958

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.39958
License (as archived)
CC0-1.0
Archived
2026-10-07T12:43:03.415Z
SHA-256 of the archived text
eabaf061414e7a5a279872f3fe2717b897caffe4bf9d111ef08b0d5a32a021d9
Stable URL of this page
https://metraipolis.com/documento/dc_46ebf63d25bc

How to cite

Better Deck or Different Judge? Evaluating Agentic Harness Gains in Corporate and Investment Banking — arXiv:2609.39958 — arXiv (en), https://arxiv.org/abs/2609.39958 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_46ebf63d25bc (Metraipolis archive dc_46ebf63d25bc, sha256 eabaf061414e7a5a279872f3fe2717b897caffe4bf9d111ef08b0d5a32a021d9)
BibTeX
@misc{dc46ebf63d25bc,
  title = {Better Deck or Different Judge? Evaluating Agentic Harness Gains in Corporate and Investment Banking — arXiv:2609.39958},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.39958},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_46ebf63d25bc; sha256 eabaf061414e7a5a279872f3fe2717b897caffe4bf9d111ef08b0d5a32a021d9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.39958. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Corporate and investment banking teams use presentations to support credit decisions and advise clients on financing and transactions. Producing these decks requires reconciling financial data, tracing sources and turning analysis into a recommendation. We retrospectively study the development of an agentic harness combining a 27B language model, financial calculations, narrative templates and validation checks. LLM judges guide engineering changes and assess the resulting decks, raising the question of whether higher scores reflect better documents or changes in grading. In shared-session text-only grading with template markers removed, five judges score the complete system 20.4 to 33.6 points out of 95 above the same model generating directly from a short prompt. Every judge scores the system higher on all seventeen development deliverables. Margins against direct Opus generation from a short prompt range from -4.7 to +0.8 points. Judges agree on broad progress across development rounds but agree less on final-deck rankings than on pooled scores. Repeated grading also shifts scores on unchanged decks, making small improvements difficult to distinguish from judge variability.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.39958 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T12:14:13.803Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).