Metraipolis › archive › dc_3eaab7723055

VisLens: Single-Pass Interpretable Visual Search for Multimodal LLMs — arXiv:2608.30705

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.30705
License (as archived)
CC0-1.0
Archived
2026-10-17T09:59:38.271Z
SHA-256 of the archived text
69ca7f676f4541ffffa5d006792722a7ab14f438bf220a897db70183de8eca41
Stable URL of this page
https://metraipolis.com/documento/dc_3eaab7723055

How to cite

VisLens: Single-Pass Interpretable Visual Search for Multimodal LLMs — arXiv:2608.30705 — arXiv (en), https://arxiv.org/abs/2608.30705 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3eaab7723055 (Metraipolis archive dc_3eaab7723055, sha256 69ca7f676f4541ffffa5d006792722a7ab14f438bf220a897db70183de8eca41)
BibTeX
@misc{dc3eaab7723055,
  title = {VisLens: Single-Pass Interpretable Visual Search for Multimodal LLMs — arXiv:2608.30705},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.30705},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3eaab7723055; sha256 69ca7f676f4541ffffa5d006792722a7ab14f438bf220a897db70183de8eca41},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.30705. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multimodal large language models (MLLMs) struggle with fine-grained Visual Search, the task of locating small or rare objects in high-resolution images. Existing remedies fall into two families: (1) Training-free methods based on attention or confidence scores are accurate but slow, since they require multiple MLLM queries per example. (2) Reinforcement Learning (RL) trained tool-use models are faster at inference but opaque, since their tool calls remain uncontrollable and hard to interpret. To overcome this, we propose \emph{VisLens} (Visual Focus via Logit Lens), a Visual Search method built on the logit lens, which decodes the semantics held in a hidden state by projecting it through the LLM head. VisLens further uses a lightweight tuned-lens that maps early hidden states into the final hidden state space, so visual tokens can be read out from early layers. These tokens are matched to target words in the query to generate a crop of the relevant region, which is fed back in alongside the original image to produce the final answer. The whole process, from decoding to the final answer, completes in a single forward pass without repeated queries. VisLens matches or exceeds prior baselines while delivering a substantial latency advantage, running $8.5$--$9.9\times$ faster than Thyme and up to $22.2\times$ faster than training-free multi-pass search methods.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-01 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2608.30705 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T09:28:48.175Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).