Metraipolis › archive › dc_d0bd4bf04b95
Contrastive Projection: Reading Transformer Internals by Differencing Logit Lenses — arXiv:2609.09902
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.09902
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T07:53:23.891Z
- SHA-256 of the archived text
605577b99e484749ee9070ebc26c6d511e6ea613987c91c2db0aab2bd3627b44- Stable URL of this page
https://metraipolis.com/documento/dc_d0bd4bf04b95
How to cite
Contrastive Projection: Reading Transformer Internals by Differencing Logit Lenses — arXiv:2609.09902 — arXiv (en), https://arxiv.org/abs/2609.09902 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d0bd4bf04b95 (Metraipolis archive dc_d0bd4bf04b95, sha256 605577b99e484749ee9070ebc26c6d511e6ea613987c91c2db0aab2bd3627b44)
BibTeX
@misc{dcd0bd4bf04b95,
title = {Contrastive Projection: Reading Transformer Internals by Differencing Logit Lenses — arXiv:2609.09902},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.09902},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d0bd4bf04b95; sha256 605577b99e484749ee9070ebc26c6d511e6ea613987c91c2db0aab2bd3627b44},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.09902. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Reading a transformer's internal states in token space is easy to do and hard to trust: a logit lens on a single hidden state is dominated, at intermediate layers, by the generic tokens the model would predict for almost any input. We read the difference instead. Subtracting two closely matched prompts' hidden states and projecting through the unembedding cancels the shared component and surfaces what separates them, an operation equivalent to reading a RepE/ActAdd steering vector through a logit lens. Built into a training-free tracer that reads at every position, sub-layer, and head and averages over designed baselines, it traces a compound- noun MLP->attention chain in Phi-2, confirmed there by activation patching, with the same distinction recovered across three architectures by readout and probe rather than by patching; it reads what retrieval surfaces for real versus fictional entities, and reads metaphor as a set of domain-to-domain mappings rather than a single figurativity feature. A cross-seed control marks the boundary: across five networks differing only in initialization, the same distinction surfaces as almost entirely different tokens (top-10 overlap 0.08). What a computation looks like in token space is network-specific; the distinction it draws is not Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-10 in cs.CL). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.09902 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T07:24:00.911Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-10?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).