Metraipolis › archive › dc_eade7346f351
From Tables to Quantified Statements: Evaluating LLM Inference Generation through Executable Verification — arXiv:2609.23966
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.23966
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T06:50:00.893Z
- SHA-256 of the archived text
9374663e0747a563d9f37532fb4a5047ac75615e0582eb779389e23c3af6ac15- Stable URL of this page
https://metraipolis.com/documento/dc_eade7346f351
How to cite
From Tables to Quantified Statements: Evaluating LLM Inference Generation through Executable Verification — arXiv:2609.23966 — arXiv (en), https://arxiv.org/abs/2609.23966 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_eade7346f351 (Metraipolis archive dc_eade7346f351, sha256 9374663e0747a563d9f37532fb4a5047ac75615e0582eb779389e23c3af6ac15)
BibTeX
@misc{dceade7346f351,
title = {From Tables to Quantified Statements: Evaluating LLM Inference Generation through Executable Verification — arXiv:2609.23966},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.23966},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_eade7346f351; sha256 9374663e0747a563d9f37532fb4a5047ac75615e0582eb779389e23c3af6ac15},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.23966. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
LLMs can generate fluent descriptions from tables, but their outputs may remain logically unsupported by the structured data. We introduce STAT-TO-TEXT, a controlled task in which LLMs generate quantified natural language inferences from statistical tables using quantified constructions such as all, some, no, and most. To evaluate these inferences, we use an LLM generated Python checker code which when executed verifies the corresponding truth conditions against the table. We compare four open-weight LLMs across model families and scales, evaluating faithfulness, logical accuracy, table coverage, and diversity. Our results show that model scale and family matter, with the largest model (GPT-OSS-120B) consistently producing the most faithful inferences without sacrificing greater table coverage and quantifier diversity, as opposed to smaller models. These findings are supported by human annotation, which shows that the automated checker closely aligns with human judgments. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-22 in cs.CL). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.23966 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T06:48:45.021Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).