Metraipolis › archive › dc_5df3e2cd64cb

Lexical Reference Frames: Interpreting LM Residual Streams in Native Semantics — arXiv:2610.22169

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.22169
License (as archived): CC0-1.0
Archived: 2026-10-16T10:02:02.659Z
SHA-256 of the archived text: 66e8292f52ee8b325095ed67db075e95fd3255585c6becaea857998398aae302
Stable URL of this page: https://metraipolis.com/documento/dc_5df3e2cd64cb

How to cite

Lexical Reference Frames: Interpreting LM Residual Streams in Native Semantics — arXiv:2610.22169 — arXiv (en), https://arxiv.org/abs/2610.22169 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5df3e2cd64cb (Metraipolis archive dc_5df3e2cd64cb, sha256 66e8292f52ee8b325095ed67db075e95fd3255585c6becaea857998398aae302)

BibTeX

@misc{dc5df3e2cd64cb,
  title = {Lexical Reference Frames: Interpreting LM Residual Streams in Native Semantics — arXiv:2610.22169},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.22169},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5df3e2cd64cb; sha256 66e8292f52ee8b325095ed67db075e95fd3255585c6becaea857998398aae302},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.22169. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

The vocabulary of a language model is its native semantics: the embedding and the unembedding fix the meaning of a state at the two ends of the residual stream. Between them, existing methods either reuse a fixed lens or treat the states as a point cloud, and neither reads a state the way the model does. We argue that every hidden state has a native reference frame, induced by the layers that read it next and adaptive to the context. Lexical reference frames approximate it by reading a state through the model's own remaining layers with a stated number of rounds of attention between positions and a typical attention output for the rest, without fitting to the model's output. No lens, with fixed directions per layer, can exceed the square root of the native frame's coherence, 0.30 and 0.35 at mid-depth of SmolLM3-3B and OLMo3-7B, whereas the first-order frame, LexFrame[1], reaches 0.97 and 0.97 and reads a state within 0.16 nats of the model's output. Its coefficients attribute predictions more faithfully than attribution patching, its lexical basis steers a prediction while sparing other tokens, and its confident readings would let an early exit skip 0.45 and 0.42 of the layers, where lenses skip almost none. In this way, lexical reference frames make every hidden state of the model readable, attributable and steerable.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.22169 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T08:44:47.699Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).