Metraipolis › archive › dc_d00c577ffd21
How High Is 0.6? Floors, Ceilings, and Headroom in Interpretability Probing — arXiv:2610.08544
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.08544
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T11:21:55.061Z
- SHA-256 of the archived text
cd00f7b97b02b35e7a7a915242ef9f7067de9eace8377a05df7eda1f58098d86- Stable URL of this page
https://metraipolis.com/documento/dc_d00c577ffd21
How to cite
How High Is 0.6? Floors, Ceilings, and Headroom in Interpretability Probing — arXiv:2610.08544 — arXiv (en), https://arxiv.org/abs/2610.08544 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d00c577ffd21 (Metraipolis archive dc_d00c577ffd21, sha256 cd00f7b97b02b35e7a7a915242ef9f7067de9eace8377a05df7eda1f58098d86)
BibTeX
@misc{dcd00c577ffd21,
title = {How High Is 0.6? Floors, Ceilings, and Headroom in Interpretability Probing — arXiv:2610.08544},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.08544},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d00c577ffd21; sha256 cd00f7b97b02b35e7a7a915242ef9f7067de9eace8377a05df7eda1f58098d86},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.08544. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Probes are the workhorse of interpretability. If a model's hidden states predict a variable, the model is said to represent it. But a probe score has no fixed meaning. An $R^2$ of 0.6 may only reflect what the input already gives away, and the same score can mean different things on different data. We propose reading every probe score against two reference points: a floor, what a declared set of simple inputs already predicts, and a ceiling, what the full input can predict. The gap between them, the headroom, is the range in which a probe can show that a model computes something beyond the simple inputs. We prove that headroom vanishes in two ways: the target stops depending on a hidden variable the model must infer, or the input stops revealing it. We test this on transformers trained for in-context meta-analysis, which must infer the hidden heterogeneity between studies to weight them correctly, and where both reference points are known. Under distribution shift, probe scores fall and prediction error rises $12$--$15\times$, yet the model recovers a similar share of the headroom, indicating that the data lost information, not the representation. We then analyze the real models. The single-cell foundation model scGPT encodes biological variability only partially. We also revisit four influential LLM probing studies, which claim that models represent geography, the state of an Othello board, truth, and the demographics of their users. Against a floor computed from the input text alone, some of these claims hold, while others are largely explained by the text itself. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.08544 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T11:15:24.841Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).