Metraipolis › archive › dc_add552ac8c36

iS-KV: Online Low-Rank KV Cache Compression via Block-Incremental SVD — arXiv:2610.02815

Source
arXiv
Original
https://arxiv.org/abs/2610.02815
License (as archived)
CC0-1.0
Archived
2026-10-05T22:49:15.623Z
SHA-256 of the archived text
f556f469cd274fdd871496df66d6757ed875a7902bb3d927cca09f2788874d83
Stable URL of this page
https://metraipolis.com/documento/dc_add552ac8c36

How to cite

iS-KV: Online Low-Rank KV Cache Compression via Block-Incremental SVD — arXiv:2610.02815 — arXiv, https://arxiv.org/abs/2610.02815 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_add552ac8c36 (Metraipolis archive dc_add552ac8c36, sha256 f556f469cd274fdd871496df66d6757ed875a7902bb3d927cca09f2788874d83)
BibTeX
@misc{dcadd552ac8c36,
  title = {iS-KV: Online Low-Rank KV Cache Compression via Block-Incremental SVD — arXiv:2610.02815},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.02815},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_add552ac8c36; sha256 f556f469cd274fdd871496df66d6757ed875a7902bb3d927cca09f2788874d83},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.02815. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Long chain-of-thought reasoning substantially increases KV-cache memory during autoregressive decoding, as every generated token introduces new key and value states and causes the cache to grow linearly with decoding length. Existing KV-cache compression methods typically control this growth through token eviction, but irreversible deletion can remove historical states that later reasoning may need to revisit. SVD-based low-rank compression provides an alternative by retaining all positions with a more compact representation. However, extending it from a fixed prompt cache to online decoding is non-trivial. Through our investigation, we find that if the basis is updated for new tokens while old tokens keep their coordinates in the old basis, the stored history drifts substantially. Based on this observation, we propose iS-KV, an online low-rank KV-cache compression method for long-horizon reasoning. iS-KV keeps a recent window exact while incrementally folding older states into bounded-rank representations. As the low-rank basis evolves, it synchronizes historical coordinates with the updated basis to maintain representation consistency. On DeepSeek-R1-Distill-Llama-8B, iS-KV achieves 82.6% accuracy at 4.06-fold persistent-KV compression, close to the original model's 83.6%. On Qwen3-8B, it achieves 89.2% accuracy at 5.64-fold compression. Under matched memory budgets, iS-KV consistently outperforms token-eviction baselines.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.02815 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-05T22:45:55.634Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).