Metraipolis › archive › dc_d4315362ccc3

When Financial Fine-tuning Fails: A Three-Level Detectability Analysis of Numerical Hallucination in Domain-Adapted Language Models — arXiv:2609.04806

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.04806
License (as archived)
CC0-1.0
Archived
2026-10-16T22:20:57.034Z
SHA-256 of the archived text
7d50ebeb190a424b66c970f53da43f38dda9563e94e3c8c8a3b8886eaa507207
Stable URL of this page
https://metraipolis.com/documento/dc_d4315362ccc3

How to cite

When Financial Fine-tuning Fails: A Three-Level Detectability Analysis of Numerical Hallucination in Domain-Adapted Language Models — arXiv:2609.04806 — arXiv (en), https://arxiv.org/abs/2609.04806 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d4315362ccc3 (Metraipolis archive dc_d4315362ccc3, sha256 7d50ebeb190a424b66c970f53da43f38dda9563e94e3c8c8a3b8886eaa507207)
BibTeX
@misc{dcd4315362ccc3,
  title = {When Financial Fine-tuning Fails: A Three-Level Detectability Analysis of Numerical Hallucination in Domain-Adapted Language Models — arXiv:2609.04806},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.04806},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d4315362ccc3; sha256 7d50ebeb190a424b66c970f53da43f38dda9563e94e3c8c8a3b8886eaa507207},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.04806. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Financial large language models are increasingly deployed for summarization of reports and disclosures, where numerical hallucination poses significant practical risks. While prior work often attributes such hallucination to insufficient numerical reasoning, this assumption has not been systematically tested under controlled fine-tuning settings. In this paper, we conduct a cost-effective, controlled study of numerical hallucination in financial summarization across three model variants: a base instruction-tuned model, a domain language-adapted model (FT-A), and a numeracy-enhanced domain model (FT-A+B+C). We introduce a three-level detectability taxonomy distinguishing between overt hallucination (currency-denominated fabrication), covert-explicit hallucination (professional-convention numbers), and covert-implicit hallucination (ungrounded quantitative claims). Our results reveal that domain fine-tuning substantially degrades numerical restraint at all detectability levels. While the Base model maintains near-zero hallucination rates (5.4\%), FT-A exhibits 82.5\% overt hallucination and FT-A+B+C reaches 98\%. Contrary to intuition, numeracy supervision amplifies rather than mitigates hallucination across all levels. We identify template injection---the insertion of memorized canonical values regardless of input content---as a primary hallucination mechanism in fine-tuned models. These findings demonstrate that numerical hallucination in financial summarization is driven by the degradation of numerical restraint through domain adaptation, not by insufficient numerical reasoning. We recommend that evaluation protocols assess hallucination across all detectability levels and that deployment practices include explicit mechanisms for grounding-aware generation or abstention.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-07 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.04806 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T22:18:11.396Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-07?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).