Metraipolis › archive › dc_a04ff09414e3
Calibrated concept profiles: making the reading of a model's neurons a measurement — arXiv:2611.09196
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.09196
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-09T22:35:56.734Z
- SHA-256 of the archived text
d0e469761df4683937c258b30604066d5fd46abf2537864b9712fd67e508bb0f- Stable URL of this page
https://metraipolis.com/documento/dc_a04ff09414e3
How to cite
Calibrated concept profiles: making the reading of a model's neurons a measurement — arXiv:2611.09196 — arXiv (en), https://arxiv.org/abs/2611.09196 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a04ff09414e3 (Metraipolis archive dc_a04ff09414e3, sha256 d0e469761df4683937c258b30604066d5fd46abf2537864b9712fd67e508bb0f)
BibTeX
@misc{dca04ff09414e3,
title = {Calibrated concept profiles: making the reading of a model's neurons a measurement — arXiv:2611.09196},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.09196},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a04ff09414e3; sha256 d0e469761df4683937c258b30604066d5fd46abf2537864b9712fd67e508bb0f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.09196. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Interpretability has largely moved from reading a model's neurons directly to reading a second model trained on the first: a probe, a sparse autoencoder, a crosscoder. The move answered real faults -- mixed meanings, thresholds without a unit, readings that did not transfer from one study to the next -- but gave up what a direct reading alone provides: nothing trained in between, the neurons themselves named, and a reading that can be repeated and compared. Here the direct reading is made a measurement. A neuron's activation varies with the prompt, so its reading is a distribution over a population of stimuli. We declare that population: every factor around a target construct is drawn at its rate in a reference corpus and logged. From the declared distribution we generate a battery of prompts, and draw it twice. From the battery we compute the calibrated concept profile, one number per neuron, stated in units of the neuron's own spread on ordinary input; the two draws give every number a noise unit. Before the profile is read, a fixed set of checks licenses the battery on the model, and the smallest change the profile can resolve across training checkpoints is measured. Realised for Python on OpenCoder-1.5B and followed across checkpoints spanning two trillion training tokens, the profile gives what the alternatives do not report: a unit on every number, a chance level for every comparison, and which neurons carry a construct at every checkpoint. The battery, the code and the data behind every result are released. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2611.09196 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-09T22:31:01.946Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).