Metraipolis › archive › dc_024f55c69278

CoVisco: Codec-Native Vision Encoder with Native Token Compression for Unified Image-Video Understanding — arXiv:2609.39924

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.39924
License (as archived)
CC0-1.0
Archived
2026-10-07T12:45:57.533Z
SHA-256 of the archived text
0f16808b432f1bd800506cc89e9bede5225575eb94324757192926e2ab7ef742
Stable URL of this page
https://metraipolis.com/documento/dc_024f55c69278

How to cite

CoVisco: Codec-Native Vision Encoder with Native Token Compression for Unified Image-Video Understanding — arXiv:2609.39924 — arXiv (en), https://arxiv.org/abs/2609.39924 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_024f55c69278 (Metraipolis archive dc_024f55c69278, sha256 0f16808b432f1bd800506cc89e9bede5225575eb94324757192926e2ab7ef742)
BibTeX
@misc{dc024f55c69278,
  title = {CoVisco: Codec-Native Vision Encoder with Native Token Compression for Unified Image-Video Understanding — arXiv:2609.39924},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.39924},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_024f55c69278; sha256 0f16808b432f1bd800506cc89e9bede5225575eb94324757192926e2ab7ef742},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.39924. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Vision-language models face a fundamental scaling bottleneck: the number of visual tokens grows with both temporal duration and spatial resolution, making long-video understanding expensive for the vision encoder and the language model. Existing methods often compress visual tokens after dense encoding, creating a mismatch between the representation used during training and the compact interface required at deployment. We present CoVisco, a codec-native vision encoder with native token compression for unified image-video understanding. By combining codec-native input support with segmented attention, CoVisco can encode long visual inputs in a single forward pass without forming dense patch-to-patch interactions across all frames. Each temporal segment is equipped with learnable abstract tokens that learn a compact segment-level representation, while fine-grained patch tokens remain available throughout the encoder. Alternating intra-segment and abstract-communication layers preserve video-level context through the abstract-token channel. A lightweight selector further exposes either abstract tokens alone or abstract tokens augmented with a runtime-selected subset of patch tokens, yielding a compact visual interface that reduces the visual context and prefill burden of downstream MLLMs while retaining fine-grained evidence when needed. Pretrained with contrastive objectives on 565M image--text pairs and 6.4M videos, CoVisco shows competitive performance on video-oriented embedding and multimodal understanding benchmarks. In the evaluated four-segment, 64-frame setting, abstract-only inference uses only 400 visual tokens while achieving video-understanding performance close to, and on some benchmarks exceeding, OneVision-Encoder. Selected patch tokens further improve fine-grained video reasoning. Project URL: this https URL

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in cs.AI).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.39924 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T12:14:14.182Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).