Metraipolis › archive › dc_e7d4c4042e52
A Lie Detector Test for Language Models: Reading Knowledge a Model Won't Reveal — arXiv:2609.21996
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.21996
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-10T01:05:38.907Z
- SHA-256 of the archived text
5bae48bf2d81e8669aaee25be696e99f0ffdc4a6a67d4b198261252b74cfc8e0- Stable URL of this page
https://metraipolis.com/documento/dc_e7d4c4042e52
How to cite
A Lie Detector Test for Language Models: Reading Knowledge a Model Won't Reveal — arXiv:2609.21996 — arXiv (en), https://arxiv.org/abs/2609.21996 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e7d4c4042e52 (Metraipolis archive dc_e7d4c4042e52, sha256 5bae48bf2d81e8669aaee25be696e99f0ffdc4a6a67d4b198261252b74cfc8e0)
BibTeX
@misc{dce7d4c4042e52,
title = {A Lie Detector Test for Language Models: Reading Knowledge a Model Won't Reveal — arXiv:2609.21996},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.21996},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e7d4c4042e52; sha256 5bae48bf2d81e8669aaee25be696e99f0ffdc4a6a67d4b198261252b74cfc8e0},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.21996. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models can hold knowledge they do not report. A model may sandbag on a capability evaluation, or answer against what it internally knows, and its outputs alone cannot tell whether it is hiding an answer or simply does not have one. We borrow the Concealed Information Test, a forensic method that identifies guilty knowledge by presenting a suspect with the true detail among plausible decoys and measuring a stronger response to the item they recognize. Our method, Probe of Internal Recognition (PIR), does the same inside a model. It presents a question with its candidate answers and reads, from the model's internal states, which candidate the model recognizes as correct. PIR is reference-free, needing no honest reference model and no labeled truth corpus. Across eight models from five families (Gemma, Qwen, Llama, Mistral, and Phi), PIR recovers the recognized answer at 0.70 to 0.87 balanced accuracy, well above the 0.28 to 0.40 unknown-item baseline and the 0.25 chance rate. It stays readable across every form of concealment we test, from prompted deception and trained sandbagging to external password-locked and circuit-broken checkpoints, with recognition between 0.85 and 0.93. When the model hides a known answer, recognition stays high. When unlearning removes the knowledge, recognition drops to the level of a question the model never knew. PIR therefore separates a model that will not answer from one that cannot, which supports sandbagging audits and unlearning verification. The signal is causal, adds information beyond black-box behavioral cues, and extends from multiple-choice questions to free-form generation. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-21 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.21996 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-10T00:23:42.483Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-21?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).