Metraipolis › archive › dc_a67cf1a587f6

VLMs Read Aloud: The Gap Between Recognizing and Using Visual Text — arXiv:2611.01517

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.01517
License (as archived): CC0-1.0
Archived: 2026-11-03T18:24:37.099Z
SHA-256 of the archived text: 32f42174ff6ed6987c265d7c0d7ab00b88d4f8132f82f53787fb1ea79a3003a3
Stable URL of this page: https://metraipolis.com/documento/dc_a67cf1a587f6

How to cite

VLMs Read Aloud: The Gap Between Recognizing and Using Visual Text — arXiv:2611.01517 — arXiv (en), https://arxiv.org/abs/2611.01517 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a67cf1a587f6 (Metraipolis archive dc_a67cf1a587f6, sha256 32f42174ff6ed6987c265d7c0d7ab00b88d4f8132f82f53787fb1ea79a3003a3)

BibTeX

@misc{dca67cf1a587f6,
  title = {VLMs Read Aloud: The Gap Between Recognizing and Using Visual Text — arXiv:2611.01517},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.01517},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a67cf1a587f6; sha256 32f42174ff6ed6987c265d7c0d7ab00b88d4f8132f82f53787fb1ea79a3003a3},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01517. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Visual-text compression renders long contexts as images and is engineered around how much of the text a vision-language model (VLM) can recover. Yet recovered text is not usable text. VLMs can transcribe the text in an image word for word and still fail to answer from it; given back their own transcript, they answer nearly as well as from the original text. We call this difference the read-use gap. Recognizing and using visual text have different resolution requirements: Qwen3-VL-8B's recognition saturates at 15-pixel characters while use catches up only at 24 pixels, and scaling from 2B to 32B raises recognition without narrowing the gap. Putting the evidence into words closes it: thinking VLMs recite the evidence in their thinking, and their own transcript, written without seeing the question, recovers most of what thinking adds. In a preregistered held-out test, the gap ranks how much eight VLMs gain from their own words over pixels (Spearman $\rho$ = 0.86), which recognition rates do not. We propose ALOUD, which spends visual resolution on finding and recognizing evidence and language tokens on using it. Without keeping the source text, it doubles the accuracy of a frozen Qwen3-VL-8B on long documents compressed 5x and 10x: at 5x, from 27.7% to 54.8%, near the original text's 58.5%, with a context smaller than the compressed pages themselves. For less than the cost of reading the original text, it leads at equal budget every method that does without it, at every level, and beats AGAR and VERA, which paste the original text back; at 5x it also beats LensVLM-9B, which is trained and keeps the original text. Visual-text compression is limited not by how much text a model can recover, but by how much it can use.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2611.01517 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T18:10:08.037Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).