Metraipolis › archive › dc_3fa010b643d1
Foresight: planning future perception in streaming VLMs without retraining — arXiv:2610.03123
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.03123
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-05T22:50:29.118Z
- SHA-256 of the archived text
e0b59e7bed8be4ca74b9080503dd1b6bb6e53d82bb28528dd11a09a79168be1a- Stable URL of this page
https://metraipolis.com/documento/dc_3fa010b643d1
How to cite
Foresight: planning future perception in streaming VLMs without retraining — arXiv:2610.03123 — arXiv, https://arxiv.org/abs/2610.03123 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3fa010b643d1 (Metraipolis archive dc_3fa010b643d1, sha256 e0b59e7bed8be4ca74b9080503dd1b6bb6e53d82bb28528dd11a09a79168be1a)
BibTeX
@misc{dc3fa010b643d1,
title = {Foresight: planning future perception in streaming VLMs without retraining — arXiv:2610.03123},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.03123},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3fa010b643d1; sha256 e0b59e7bed8be4ca74b9080503dd1b6bb6e53d82bb28528dd11a09a79168be1a},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.03123. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Existing streaming vision-language models (VLMs) continuously perceive and reason over visual streams, but their computational pathways remain fixed throughout inference. Consequently, they cannot adapt computation to evolving scene dynamics, where different future events demand different levels and forms of perception. We show that streaming VLMs inherently possess the ability to anticipate the immediate future, and leverage this capability to dynamically configure future computation in a training-free manner. Realizing such anticipatory computation, however, is very challenging: future anticipation must be sufficiently reliable to guide computation, planning must run concurrently with streaming inference, and online reconfiguration must incur negligible overhead. To address these challenges, we introduce FORESIGHT, a dual-stream architecture comprising two Siamese LLMs with shared weights, input encoders, and KV cache. The first LLM continuously processes incoming tokens, while the second runs ahead of the stream to anticipate future context, plan future computation, and generate task responses without interrupting streaming inference. Each plan decides when to reason next, what to check then, and how densely to sample, keeping transient evidence separate from persistent control. The resulting computation plan is executed online through an efficient reconfiguration protocol with schemaguided decoding and lightweight diff-based updates, enabling dynamic adaptation with low overhead. With a frozen Qwen3-VL-8B backbone, FORESIGHT achieves 23.0 mean joint F1 on OmniPro Online evaluation beating strongest trained baseline by 9.5%, while improving the backbone by 6.7 on StreamingBench and 15.4 on OVO-Bench, with the largest gain of 18.7 when evidence arrives later in the video stream. Our source code will be made publicly available. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.03123 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-05T22:45:55.832Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).