Metraipolis › archive › dc_503cfcf6525d

Latent-Lens: Visual Perception in Small Language Models Through Latent Communication — arXiv:2610.41751

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.41751
License (as archived): CC0-1.0
Archived: 2026-11-02T07:01:14.931Z
SHA-256 of the archived text: 6356c7778347eadd7b50a0f93dafc61f0eff9240156f5261a82cb845dbe1dd6f
Stable URL of this page: https://metraipolis.com/documento/dc_503cfcf6525d

How to cite

Latent-Lens: Visual Perception in Small Language Models Through Latent Communication — arXiv:2610.41751 — arXiv (en), https://arxiv.org/abs/2610.41751 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_503cfcf6525d (Metraipolis archive dc_503cfcf6525d, sha256 6356c7778347eadd7b50a0f93dafc61f0eff9240156f5261a82cb845dbe1dd6f)

BibTeX

@misc{dc503cfcf6525d,
  title = {Latent-Lens: Visual Perception in Small Language Models Through Latent Communication — arXiv:2610.41751},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.41751},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_503cfcf6525d; sha256 6356c7778347eadd7b50a0f93dafc61f0eff9240156f5261a82cb845dbe1dd6f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41751. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Bridging independent vision-language models (VLMs) and small language models (SLMs) in a modular multi-agent system typically forces them to exchange information as discrete text, which creates a lossy bottleneck for spatial and fine-grained details. This constraint is especially acute for SLMs, whose limited capacity makes them more reliant on the quality of the input signal. Recent latent-communication methods exchange states between language models. How a VLM can pass visual information to a separately pretrained text-only SLM though remains unexplored. We propose \textbf{Latent-Lens}, a lightweight codec that bridges the representation gap between a frozen VLM and a frozen SLM directly in latent space, enabling them to communicate purely through hidden states. The codec consists of a domain encoder that projects hidden states from the VLM into the SLM's embedding space, and an attention decoder formalised as a LoRA that teaches the SLM to attend to these visual prefixes, while training only ${\sim}1.6$M parameters (0.2\% of the combined model size). On visual question answering (CLEVR, GQA), Latent-Lens achieves $81.0\pm7.9$\% and $65.4\pm5.4$\%, compared with $58.3\pm5.7$\% and $61.8\pm1.0$\% for text communication with the same LoRA budget (+30.2--69.8pp relative to frozen text communication), while on image captioning (Flickr8k) it improves over the same baseline by +16.5 (+18.8 frozen). We also evaluate Latent-Lens with TinyLlama-1.1B and InternVL2-1B, which use different architectures, vocabularies, and hidden dimensions.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.41751 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-02T06:56:58.406Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).