Metraipolis › archive › dc_ebd8aff4106d

When Text and Numbers Disagree: Evidence Arbitration in Large Language Models — arXiv:2608.20116

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.20116
License (as archived)
CC0-1.0
Archived
2026-10-18T11:39:23.568Z
SHA-256 of the archived text
bd0b4e431e26956f2f2253f708ea09c4418bd1eabd957dfe50a77485397f40d5
Stable URL of this page
https://metraipolis.com/documento/dc_ebd8aff4106d

How to cite

When Text and Numbers Disagree: Evidence Arbitration in Large Language Models — arXiv:2608.20116 — arXiv (en), https://arxiv.org/abs/2608.20116 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ebd8aff4106d (Metraipolis archive dc_ebd8aff4106d, sha256 bd0b4e431e26956f2f2253f708ea09c4418bd1eabd957dfe50a77485397f40d5)
BibTeX
@misc{dcebd8aff4106d,
  title = {When Text and Numbers Disagree: Evidence Arbitration in Large Language Models — arXiv:2608.20116},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.20116},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ebd8aff4106d; sha256 bd0b4e431e26956f2f2253f708ea09c4418bd1eabd957dfe50a77485397f40d5},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.20116. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) are increasingly used in settings where textual summaries, numerical observations, and external tool outputs may provide conflicting evidence. We study how LLMs arbitrate between such sources when they support opposing decisions. To do so, we introduce a controlled synthetic benchmark in which latent risk trajectories generate both numerical time series and natural language summaries, allowing us to construct conflicts where exactly one evidence source is aligned with the ground-truth label. This design lets us independently manipulate modality, temporal recency, source reliability, and evidence provenance. Across open-weight instruction-tuned models, we find that arbitration behaviour is systematic rather than random: models exhibit distinct text-versus-number preferences, follow temporal recency more consistently than explicit reliability cues, and can over-rely on external forecasts even when they conflict with direct contextual evidence. These results suggest that current LLMs often rely on heuristic arbitration strategies when integrating heterogeneous evidence, highlighting a failure mode for tool-augmented decision systems.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-21 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.20116 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T11:38:27.928Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-21?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).