Metraipolis › archive › dc_67bf994210ae
LGMT: A Tool for Testing the Logical Reasoning Reliability of LLMs — arXiv:2610.32627
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.32627
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-26T19:50:43.529Z
- SHA-256 of the archived text
f679e1df24f6a7dee1a7264c7a82fc0750b6534d85431cd1cdcbc730bf1edc3d- Stable URL of this page
https://metraipolis.com/documento/dc_67bf994210ae
How to cite
LGMT: A Tool for Testing the Logical Reasoning Reliability of LLMs — arXiv:2610.32627 — arXiv (en), https://arxiv.org/abs/2610.32627 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_67bf994210ae (Metraipolis archive dc_67bf994210ae, sha256 f679e1df24f6a7dee1a7264c7a82fc0750b6534d85431cd1cdcbc730bf1edc3d)
BibTeX
@misc{dc67bf994210ae,
title = {LGMT: A Tool for Testing the Logical Reasoning Reliability of LLMs — arXiv:2610.32627},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.32627},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_67bf994210ae; sha256 f679e1df24f6a7dee1a7264c7a82fc0750b6534d85431cd1cdcbc730bf1edc3d},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.32627. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large Language Models (LLMs) increasingly serve as reasoning components in modern software systems, yet a model that answers a logical query correctly can contradict itself when the same problem is restated in an equivalent form. Static benchmarks evaluate queries in isolation and therefore fail to expose such inconsistencies. To address this limitation, we present LGMT, an open-source tool that detects reasoning inconsistencies through metamorphic relations grounded in first-order logic. Given a source test case in premise-conclusion form, LGMT generates logically equivalent follow-up cases and checks whether the model's entailment predictions agree across the pair, without ground-truth labels. The tool provides 20 built-in metamorphic relations, supports custom ones verified by the Z3 theorem prover, and runs from a CLI or a web studio, including an offline mock mode requiring no API key. Across six widely used LLMs on a bundled benchmark of 3,091 metamorphic groups, LGMT reveals inconsistency rates ranging from 20.84% to 29.44%. LGMT is publicly available on GitHub at this https URL , with a demonstration video at this https URL . Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.32627 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-26T19:45:33.326Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).