Metraipolis › archive › dc_ba7de5041845
Geometry of Confabulation and Refusal in Small Language Models — arXiv:2611.01272
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.01272
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T18:24:36.191Z
- SHA-256 of the archived text
f888eaaea110b3d7fd88c8daead700564adf3ba965174a9d1d1a51abbac4e9a5- Stable URL of this page
https://metraipolis.com/documento/dc_ba7de5041845
How to cite
Geometry of Confabulation and Refusal in Small Language Models — arXiv:2611.01272 — arXiv (en), https://arxiv.org/abs/2611.01272 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ba7de5041845 (Metraipolis archive dc_ba7de5041845, sha256 f888eaaea110b3d7fd88c8daead700564adf3ba965174a9d1d1a51abbac4e9a5)
BibTeX
@misc{dcba7de5041845,
title = {Geometry of Confabulation and Refusal in Small Language Models — arXiv:2611.01272},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.01272},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ba7de5041845; sha256 f888eaaea110b3d7fd88c8daead700564adf3ba965174a9d1d1a51abbac4e9a5},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01272. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Faced with an unanswerable question, small language models can either refuse or confabulate: confidently produce a wrong answer. Can the hidden state before the first generated token provide a clue? Can we flip confabulations into refusals? I introduce ConfabQA, a 784-item benchmark whose obscure-pre-cutoff category breaks the typical correctness/knowledge-cutoff confound and map the geometry of the hidden state with linear probes, a sparse autoencoder (SAE), and direct optimization of intervention directions. On Qwen3-1.7B and Gemma 2 2B the probe separates upcoming refusals from confabulations well beyond prompt features (+7.4/+16.3 pp); on Llama 3.2 3B the ConfabQA margin is within fold-to-fold spread, but a direct refusal probe on PopQA and TriviaQA recovers +18-25 pp. On Qwen I closely inspect three candidate directions for the refusal vector: probe-recovered, Qwen-Scope SAE feature, and directly optimized. Although the three directions are pairwise 50-81 degrees apart, a push at or below the state's own norm along any of them flips most confabulations into refusal openings dominated by the "As..." family. Forcing the opener without touching the hidden state yields the same one-in-three judged-refusal rate as any push. The opposite push collapses refusals mostly into confabulations, yet leaves originally correct answers intact, so the direction gates the opening speech act rather than the knowledge behind it. Confabulations do not resolve into a concentrated feature in a dictionary or a token signature on Qwen3-1.7B. Scoring every probe on the margin over the strongest of four prompt-feature baselines extracts the hidden state's own contribution. Correctness signal is model-dependent and portable only for Llama 3.2 3B: 81% of the probe's margin over test majority survives transfer between ConfabQA, PopQA, and TriviaQA. Benchmark, judge, and code are released. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2611.01272 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T18:10:08.034Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).