Metraipolis › archive › dc_ab9800fc867a
Automated Testing of LLM-Based Post Hoc Explainers Using Model Checking as an Oracle — arXiv:2608.30581
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.30581
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T07:44:47.132Z
- SHA-256 of the archived text
03fbebf256df8f59dd510b9668bd3ccecb471beb353a7bc50131b833b0405399- Stable URL of this page
https://metraipolis.com/documento/dc_ab9800fc867a
How to cite
Automated Testing of LLM-Based Post Hoc Explainers Using Model Checking as an Oracle — arXiv:2608.30581 — arXiv (en), https://arxiv.org/abs/2608.30581 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ab9800fc867a (Metraipolis archive dc_ab9800fc867a, sha256 03fbebf256df8f59dd510b9668bd3ccecb471beb353a7bc50131b833b0405399)
BibTeX
@misc{dcab9800fc867a,
title = {Automated Testing of LLM-Based Post Hoc Explainers Using Model Checking as an Oracle — arXiv:2608.30581},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.30581},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ab9800fc867a; sha256 03fbebf256df8f59dd510b9668bd3ccecb471beb353a7bc50131b833b0405399},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.30581. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models (LLMs) are used as post hoc explainers of sequential decision-making policies, producing natural-language explanations of why an action was chosen. However, LLMs often generate plausible but incorrect statements, and no existing approach systematically tests whether such explanations are faithful to the underlying environment. Two classic software testing challenges stand in the way: there is no oracle for the correctness of an explanation, and the test inputs, natural language queries about a policy's behavior, lack the structure needed for systematic test case generation. We address both. Probabilistic model checking provides the test oracle, computing exact reference results against which LLM answers are graded automatically. A taxonomy of post hoc query categories structures the input space around the environment-level facts from which policy explanations are composed; test cases generated from it are prioritized by question-specific diagnostic difficulty scores. Across seven MDP environments, the testing separates three open-weight LLMs: a reasoning model passes 85% of test cases, a mid-size model 70%, and a 1B model falls below the random baseline, while prioritization surfaces significantly harder cases than random selection. Our results indicate how trustworthy LLM-generated explanations are in model-free settings, where the same LLMs are used but no oracle exists to verify them. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2608.30581 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T07:28:26.616Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).