Metraipolis › archive › dc_ec2f07e70b76
Beyond Accuracy: A Multidimensional Evaluation of Statistical Reasoning in Large Language Models — arXiv:2608.03038
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.03038
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-31T12:30:54.416Z
- SHA-256 of the archived text
d3ca2b31bf58ac944aa8bc7e70d1f2dec440551b14ac3f280c0b4daa27ebc42d- Stable URL of this page
https://metraipolis.com/documento/dc_ec2f07e70b76
How to cite
Beyond Accuracy: A Multidimensional Evaluation of Statistical Reasoning in Large Language Models — arXiv:2608.03038 — arXiv (en), https://arxiv.org/abs/2608.03038 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ec2f07e70b76 (Metraipolis archive dc_ec2f07e70b76, sha256 d3ca2b31bf58ac944aa8bc7e70d1f2dec440551b14ac3f280c0b4daa27ebc42d)
BibTeX
@misc{dcec2f07e70b76,
title = {Beyond Accuracy: A Multidimensional Evaluation of Statistical Reasoning in Large Language Models — arXiv:2608.03038},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.03038},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ec2f07e70b76; sha256 d3ca2b31bf58ac944aa8bc7e70d1f2dec440551b14ac3f280c0b4daa27ebc42d},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.03038. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Statistical reasoning is multidimensional, yet evaluations of large language models (LLMs) typically emphasize response accuracy while overlooking how models construct and communicate statistical explanations. This study demonstrates the value of a multidimensional evaluation by combining response accuracy, response behavior, structural topic modeling, and lexical similarity analysis. The framework is applied to explanations generated by 15 current-generation LLMs responding to 90 questions drawn from four statistics examinations spanning high school, undergraduate, and graduate levels. Accuracy varied substantially across models, ranging from 55\% to 78\%. In contrast, structural topic modeling revealed a common conceptual organization of statistical reasoning across all models, while lexical similarity analysis identified modest but consistent vendor-specific differences in explanatory style. Models developed by the same vendor (e.g. Anthropic, OpenAI) produced explanations that were slightly more similar than models from different vendors. These findings demonstrate that statistical reasoning in contemporary LLMs cannot be characterized by accuracy alone and illustrate how complementary analyses of response behavior and model-generated explanations provide a more comprehensive evaluation of statistical reasoning in generative AI. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-05 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Applications (stat.AP). Pagina del lavoro: https://arxiv.org/abs/2608.03038 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-31T12:17:37.733Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-05?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).