Metraipolis › archive › dc_cefc1e2bb069

OmniKVQuant: KV Cache Quantization for Omni-LLMs — arXiv:2609.11582

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.11582
License (as archived)
CC0-1.0
Archived
2026-10-16T06:35:44.222Z
SHA-256 of the archived text
3871a62f3c7424b0fae03d8c047bd45aa255bd1bea8fd3bae77d40260b053108
Stable URL of this page
https://metraipolis.com/documento/dc_cefc1e2bb069

How to cite

OmniKVQuant: KV Cache Quantization for Omni-LLMs — arXiv:2609.11582 — arXiv (en), https://arxiv.org/abs/2609.11582 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cefc1e2bb069 (Metraipolis archive dc_cefc1e2bb069, sha256 3871a62f3c7424b0fae03d8c047bd45aa255bd1bea8fd3bae77d40260b053108)
BibTeX
@misc{dccefc1e2bb069,
  title = {OmniKVQuant: KV Cache Quantization for Omni-LLMs — arXiv:2609.11582},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.11582},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cefc1e2bb069; sha256 3871a62f3c7424b0fae03d8c047bd45aa255bd1bea8fd3bae77d40260b053108},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.11582. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

As Omni-modal large language models (Omni-LLMs) take in audio, video and text together, their KV cache memory cost grows. KV cache quantization is the de facto approach in text-only LLMs, but its application to Omni-LLMs remains unexplored. In this paper, we analyze how TurboQuant, a representative rotation-based KV cache quantization method, behaves on multimodal caches and identify two critical issues: temporal key drift and heterogeneous value geometry. To address these, we propose OmniKVQuant, a training-free framework that (i) sets the key quantization range over each short window of the input stream; and (ii) rotates values separately per modality. On Qwen2.5-Omni and Qwen3-Omni, OmniKVQuant enables 2-bit KV caches while substantially preserving performance across seven audio-visual benchmarks. We further provide a fused Triton decode kernel that unpacks the 2-bit cache during attention, so no dense FP16 cache is ever built. Code: this https URL

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-11 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2609.11582 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T06:06:28.795Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).