Metraipolis › archive › dc_4f920cdc72fe

The Corroboration Illusion: When More News Makes LLM Forecasts Less True — arXiv:2609.22246

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.22246
License (as archived)
CC0-1.0
Archived
2026-10-09T06:49:25.888Z
SHA-256 of the archived text
85c2c6ff4b105a0d7504ef9036084c8babb45bf49df864f63adab2dcb690ace9
Stable URL of this page
https://metraipolis.com/documento/dc_4f920cdc72fe

How to cite

The Corroboration Illusion: When More News Makes LLM Forecasts Less True — arXiv:2609.22246 — arXiv (en), https://arxiv.org/abs/2609.22246 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4f920cdc72fe (Metraipolis archive dc_4f920cdc72fe, sha256 85c2c6ff4b105a0d7504ef9036084c8babb45bf49df864f63adab2dcb690ace9)
BibTeX
@misc{dc4f920cdc72fe,
  title = {The Corroboration Illusion: When More News Makes LLM Forecasts Less True — arXiv:2609.22246},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.22246},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4f920cdc72fe; sha256 85c2c6ff4b105a0d7504ef9036084c8babb45bf49df864f63adab2dcb690ace9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.22246. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) are increasingly used to forecast real-world events by retrieving and reasoning over news. We show that this dependence on an open, crawlable news corpus creates a new attack surface: an adversary who can merely publish articles--without access to the retriever, the model, or the user's queries--can systematically move the forecaster's output probabilities. We formalize news-corpus poisoning of probabilistic forecasters, a threat model distinct from prior RAG poisoning, which targets factual answers or opinion polarity rather than calibrated probabilities. We evaluate the attack on 500 resolved ForecastBench questions against a 17.4M-article Common Crawl News corpus with a strict crawl-date cutoff, using three retrieval-augmented forecasters built on open 7-8B models. A single LLM-written article per question flips 56% of forecasts across the 0.5 boundary; five articles flip 69-73% and shift probabilities by +0.13 to +0.22 net of a neutral-article placebo, degrading the Brier score from 0.18 to 0.37. The effect is monotone in the number, retrieval rank, query similarity, and context share of injected articles, transfers across model families, and is unaffected by the claimed publisher. We then evaluate three natural defenses--source allow-lists, isolate-then-aggregate forecasting, and perplexity filtering--and show that each has a cheap bypass: spoofed publishers, majority poisoning, and higher-temperature generation, respectively. Our results indicate that probabilistic LLM judgments inherit the full fragility of the information supply chain they consume.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-22 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.22246 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T06:48:44.915Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).