Metraipolis › archive › dc_8ff887e89de2
Reading Between the Layers: Cross-Modal Alignment and Representational Hierarchy in Vision Transformers — arXiv:2610.32463
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.32463
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-26T19:53:37.973Z
- SHA-256 of the archived text
9f30df88d8700ab5db20a3a4c4cfe46f27f4f80a6ee1e1077a74ef4ac8cd2e5b- Stable URL of this page
https://metraipolis.com/documento/dc_8ff887e89de2
How to cite
Reading Between the Layers: Cross-Modal Alignment and Representational Hierarchy in Vision Transformers — arXiv:2610.32463 — arXiv (en), https://arxiv.org/abs/2610.32463 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8ff887e89de2 (Metraipolis archive dc_8ff887e89de2, sha256 9f30df88d8700ab5db20a3a4c4cfe46f27f4f80a6ee1e1077a74ef4ac8cd2e5b)
BibTeX
@misc{dc8ff887e89de2,
title = {Reading Between the Layers: Cross-Modal Alignment and Representational Hierarchy in Vision Transformers — arXiv:2610.32463},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.32463},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8ff887e89de2; sha256 9f30df88d8700ab5db20a3a4c4cfe46f27f4f80a6ee1e1077a74ef4ac8cd2e5b},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.32463. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
While large vision-language models excel at optical character recognition, it remains unclear how their underlying vision encoders process text and its meaning, and whether they support anything resembling reading comprehension. We systematically study transformer-based vision encoders, using cross-modal alignment between rendered text and visual concepts together with probing and retrieval-based analyses as a controlled window into these broader capabilities. Our layer-wise analyses span orthographic, grammatical, and semantic organization, and reveal that strong cross-modal mapping emerges only in the deepest layers, after unimodal object and text recognition have already developed, coinciding with a progression from surface-level to semantic representations. Although this hierarchy is strongest in contrastively trained models, larger self-supervised models and text-rich training data partially support similar behaviors without explicit language supervision. Together, our results suggest that increasing model and training-data scale, as well as text-rich training data, can partially compensate for the absence of language supervision. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.32463 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-26T19:45:48.364Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).