Metraipolis › archive › dc_aacd49393fdc

VFold: Symmetry-Aware Cross-Layer Value Cache Compression — arXiv:2610.12338

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.12338
License (as archived)
CC0-1.0
Archived
2026-10-09T06:13:30.221Z
SHA-256 of the archived text
6c69c07ed2d18c64694cd7f3eb02c14f34db52ffc69ffa5d4c1031247ada83b4
Stable URL of this page
https://metraipolis.com/documento/dc_aacd49393fdc

How to cite

VFold: Symmetry-Aware Cross-Layer Value Cache Compression — arXiv:2610.12338 — arXiv (en), https://arxiv.org/abs/2610.12338 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_aacd49393fdc (Metraipolis archive dc_aacd49393fdc, sha256 6c69c07ed2d18c64694cd7f3eb02c14f34db52ffc69ffa5d4c1031247ada83b4)
BibTeX
@misc{dcaacd49393fdc,
  title = {VFold: Symmetry-Aware Cross-Layer Value Cache Compression — arXiv:2610.12338},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.12338},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_aacd49393fdc; sha256 6c69c07ed2d18c64694cd7f3eb02c14f34db52ffc69ffa5d4c1031247ada83b4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.12338. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

While caching key-value (KV) states accelerates Large Language Model (LLM) decoding, this cache can dominate memory usage at long context lengths. One solution is to compress this memory by exploiting inter-layer cache similarities. However, most existing techniques necessitate architectural changes to LLMs and incur substantial overhead. In this work, we propose a symmetry-aware value cache merging strategy that reduces cache memory while avoiding both harmful performance degradation and architectural overhead during decoding. Furthermore, we show that this approach can be exploited alongside existing cache compression techniques, composing with high-ratio quantization or key cache pruning to reach compression ratios that neither method reaches alone, with minimal additional cost. Ultimately, our findings reveal a major source of underutilized capacity in the value cache, offering a simple yet highly effective direction for scaling context windows under memory constraints.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.12338 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T05:47:15.403Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).