Metraipolis › archive › dc_86bcba2837a7

Computation Over Geometry: Meaning Identity Is Computed, Not Shipped in the Embeddings — arXiv:2609.28290

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.28290
License (as archived)
CC0-1.0
Archived
2026-10-08T10:05:17.340Z
SHA-256 of the archived text
dc17d05da27006e52c8a89ba35e88c0052d4878358a8c200ad1240e77ac25d9d
Stable URL of this page
https://metraipolis.com/documento/dc_86bcba2837a7

How to cite

Computation Over Geometry: Meaning Identity Is Computed, Not Shipped in the Embeddings — arXiv:2609.28290 — arXiv (en), https://arxiv.org/abs/2609.28290 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_86bcba2837a7 (Metraipolis archive dc_86bcba2837a7, sha256 dc17d05da27006e52c8a89ba35e88c0052d4878358a8c200ad1240e77ac25d9d)
BibTeX
@misc{dc86bcba2837a7,
  title = {Computation Over Geometry: Meaning Identity Is Computed, Not Shipped in the Embeddings — arXiv:2609.28290},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.28290},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_86bcba2837a7; sha256 dc17d05da27006e52c8a89ba35e88c0052d4878358a8c200ad1240e77ac25d9d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.28290. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Meaning identity (whether two sentences say the same thing after wording changes) is treated in retrieval and RAG as a geometric fact about independently encoded sentence vectors. We show that, for frozen off-the-shelf encoders and language models, it is not: identity is computed when both sentences share one forward pass, and is not a property of the embedding geometry those systems ship. On overlap-matched PAWS-X, purpose-built encoders (BGE, E5, GTE, MiniLM, E5-Mistral-7B) reach English confirm AUC only 0.55-0.65 (dense peak 0.70). Independently encoded last-token states of Llama 3, Mistral, and Qwen do no better; late fusion of the two vectors stays near chance. The same probe on a joint forward pass reaches 0.90-0.96 from 1.5B to 32B, collapses under partner shuffle, is mid-depth, saturates near 0.94 by 3B, and appears more weakly in GPT-2 XL (0.76). The gap holds beyond Llama-style models on other causal LMs, bidirectional encoders (DeBERTa, RoBERTa), and encoder-decoders (Flan-T5, T5, BART). Fixed or linear readers over frozen independent encodings never unlock identity; nonlinear pair readers recover part of it only on the full 49k-pair PAWS train split (0.68-0.87). Off-the-shelf rerankers split: BGE-reranker-large reaches 0.94, while MS-MARCO and Jina stay at 0.55-0.64. Independently trained families compute the same relation and a 1.5B joint reader can distill it from unlabelled teacher scores, while no linear function of the teachers own independent vectors can. Bi-encoders can be fine-tuned to fit PAWS (0.87-0.93), but transfer and STS-B suffer. Cosine compares wording neighbourhoods; identity is a cheap computed operator, not a property of either sentence vector.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-24 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.28290 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T09:11:49.661Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).