Metraipolis › archive › dc_7e216f364262
ShallowStream: Index Shallow then Answer Deep for Streaming Video Understanding — arXiv:2609.02780
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.02780
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T04:17:06.651Z
- SHA-256 of the archived text
03872a705d0c67bb4b86ff219032a6c7da1cad0c415772ddb9a4c371b2c78907- Stable URL of this page
https://metraipolis.com/documento/dc_7e216f364262
How to cite
ShallowStream: Index Shallow then Answer Deep for Streaming Video Understanding — arXiv:2609.02780 — arXiv (en), https://arxiv.org/abs/2609.02780 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7e216f364262 (Metraipolis archive dc_7e216f364262, sha256 03872a705d0c67bb4b86ff219032a6c7da1cad0c415772ddb9a4c371b2c78907)
BibTeX
@misc{dc7e216f364262,
title = {ShallowStream: Index Shallow then Answer Deep for Streaming Video Understanding — arXiv:2609.02780},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.02780},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7e216f364262; sha256 03872a705d0c67bb4b86ff219032a6c7da1cad0c415772ddb9a4c371b2c78907},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.02780. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Streaming video understanding is a critical capability for real-world applications, including embodied intelligence, autonomous driving, industrial monitoring, surveillance and early warning, and wearable assistants. However, processing continuous video streams with multimodal large language models (MLLMs) is computationally expensive. Existing efforts have explored reducing streaming overhead through visual token pruning, token merging, quantization, on-demand frame retrieval, and context offloading. However, most existing methods overlook the dimension of model depth. Repeatedly executing full-depth MLLM prefill over incoming frames is prohibitively expensive, incurring substantial computational overhead and causing the KV cache to grow at a rate directly proportional to the prefill depth. To address these challenges, we propose ShallowStream, a novel framework that leverages the shallow layers of an MLLM to simultaneously perform frame encoding and retrieval index building. During stream processing, ShallowStream maintains an always-on lightweight index using the KV cache of shallow layers. During query-time answering, we leverage the attention scores generated by the shallow layers to score context frames and employ a diversity-aware selection strategy to retrieve precise and comprehensive evidence. ShallowStream achieves performance on par with the strongest existing streaming methods, while reducing per-frame prefill latency and 10-second end-to-end latency by up to 52.1x and 11.9x, respectively. Our code is available at this https URL . Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-03 in cs.CL). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.02780 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T03:26:49.843Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-03?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).