Metraipolis › archive › dc_1642e2fb9d2d
Stress-Testing the Robustness of LLM-Based Causal Inference — arXiv:2611.03656
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.03656
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-04T18:53:44.925Z
- SHA-256 of the archived text
53fdbda940b907bb38ea4439c768b5f4f5709b856b714883af943fb99ed89144- Stable URL of this page
https://metraipolis.com/documento/dc_1642e2fb9d2d
How to cite
Stress-Testing the Robustness of LLM-Based Causal Inference — arXiv:2611.03656 — arXiv (en), https://arxiv.org/abs/2611.03656 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_1642e2fb9d2d (Metraipolis archive dc_1642e2fb9d2d, sha256 53fdbda940b907bb38ea4439c768b5f4f5709b856b714883af943fb99ed89144)
BibTeX
@misc{dc1642e2fb9d2d,
title = {Stress-Testing the Robustness of LLM-Based Causal Inference — arXiv:2611.03656},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.03656},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_1642e2fb9d2d; sha256 53fdbda940b907bb38ea4439c768b5f4f5709b856b714883af943fb99ed89144},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.03656. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Why do some of the world's most powerful Artificial Intelligence (AI) systems still struggle to reason about cause and effect? Causal discovery, the ability to discern the underlying drivers of a system, is fundamental to scientific inquiry and has traditionally relied on structured data and statistical methods. Recently, Large Language Models (LLMs) have been explored as an alternative paradigm for causal inference, leveraging their broad knowledge and reasoning capabilities. Yet, despite their linguistic fluency, LLMs often rely on surface-level pattern recognition rather than genuine causal logic. While it is known that LLMs stumble in this domain, we lack a precise map of where, why, and how significantly these failures occur. In this paper, we propose a systematic stress-test framework across six critical dimensions: model scale, causal graph complexity, prompting strategy, metadata quality, data quality, and sensitivity to uncertainty. Our findings reveal a significant "optimism bias" in LLMs; models consistently overestimate causal links, leading to low precision across the board. Furthermore, we demonstrate that traditional evaluation metrics can be deceptive: while reasoning-heavy models lead in performance, the highest scores often occur when uncertainty in inference is overlooked. Once uncertainty is integrated into the evaluation, performance drops, providing a more honest reflection of causal reliability. We also find that, while scaling model size improves outcomes, these gains are fragile; performance degrades sharply as the quality of metadata decreases. This work serves as a practical guide to the blind spots of LLMs, offering a clear-eyed assessment of when these tools can be integrated into a causal discovery pipeline and when they are likely to lead researchers astray. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2611.03656 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-04T18:35:37.384Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).