Metraipolis › archive › dc_558fa652fa2e

FOCUS: Closed-Loop Attention Feedback for Efficient Vision-Language Understanding — arXiv:2611.01438

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.01438
License (as archived): CC0-1.0
Archived: 2026-11-03T10:22:14.685Z
SHA-256 of the archived text: 0f19faaf0993a2976447c53fbcfa300f1713b2ba7ba9fb7e9a147ca364bf1d01
Stable URL of this page: https://metraipolis.com/documento/dc_558fa652fa2e

How to cite

FOCUS: Closed-Loop Attention Feedback for Efficient Vision-Language Understanding — arXiv:2611.01438 — arXiv (en), https://arxiv.org/abs/2611.01438 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_558fa652fa2e (Metraipolis archive dc_558fa652fa2e, sha256 0f19faaf0993a2976447c53fbcfa300f1713b2ba7ba9fb7e9a147ca364bf1d01)

BibTeX

@misc{dc558fa652fa2e,
  title = {FOCUS: Closed-Loop Attention Feedback for Efficient Vision-Language Understanding — arXiv:2611.01438},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.01438},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_558fa652fa2e; sha256 0f19faaf0993a2976447c53fbcfa300f1713b2ba7ba9fb7e9a147ca364bf1d01},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01438. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Processing every video frame through Vision-Language Models (VLMs) adds significant inference cost and limits their ability to comprehend real-time video. Existing approaches either sub-sample frames uniformly, ignoring content and missing important events, or prune visual tokens inside the VLM to cut per-call cost while still invoking the model on every sampled frame. We introduce FOCUS, a training-free three-layer framework that reduces the number of invocations and steers each one towards what matters. Temporal gating measures inter-frame dissimilarity in the patch-embedding space of a smaller VLM from the same family, triggering burst captures only when the scene changes in a semantically meaningful way. Patch-level dissimilarity is then accumulated across the burst and fused with a top-down attention prior into a spatial focus map that encodes where motion and semantic relevance coincide. That map calibrates the VLM's value cache during generation, progressively amplifying vision tokens in task-relevant regions and attenuating off-target ones, without removing any token from the context. The loop closes after each response: text-to-vision cross-attention is extracted from the decoder and fed back as the prior for the next burst, letting the system converge on semantically important regions over successive observations. We evaluate FOCUS on the 500-video EgoSchema subset in streaming mode, with a single Qwen3-VL-8B backbone shared by every method. It reaches 74.8% accuracy using 25.4 VLM calls per clip, against 74.6% with 55 calls for uniform sampling and 74.2% with 55 calls for BOLT. On egocentric video captioning across four domains, it matches the best baseline's quality while issuing 57% fewer VLM invocations than uniform sampling.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2611.01438 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:19:49.819Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).