Metraipolis › archive › dc_3b96ea220bc8

ZipFusion: Fusing Lossless KV Decompression with Attention for Efficient LLM Decoding — arXiv:2610.33719

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.33719
License (as archived)
CC0-1.0
Archived
2026-10-27T08:14:33.916Z
SHA-256 of the archived text
cc2d89f38932e02d74c03c7644130fd2b1ab4689950d43a836722b030ecf7055
Stable URL of this page
https://metraipolis.com/documento/dc_3b96ea220bc8

How to cite

ZipFusion: Fusing Lossless KV Decompression with Attention for Efficient LLM Decoding — arXiv:2610.33719 — arXiv (en), https://arxiv.org/abs/2610.33719 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3b96ea220bc8 (Metraipolis archive dc_3b96ea220bc8, sha256 cc2d89f38932e02d74c03c7644130fd2b1ab4689950d43a836722b030ecf7055)
BibTeX
@misc{dc3b96ea220bc8,
  title = {ZipFusion: Fusing Lossless KV Decompression with Attention for Efficient LLM Decoding — arXiv:2610.33719},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.33719},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3b96ea220bc8; sha256 cc2d89f38932e02d74c03c7644130fd2b1ab4689950d43a836722b030ecf7055},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.33719. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

The key-value (KV) cache places growing demands on GPU memory capacity and bandwidth as large language models serve longer contexts and more concurrent requests. Its footprint limits concurrency, while repeated KV reads during attention can make decoding memory-bandwidth bound. Prior lossless KV cache compression methods reduce transfers between devices but restore the uncompressed KV cache in GPU memory before attention. They therefore leave these repeated KV reads unchanged and require additional work and memory for decompression. We present ZipFusion, which enables attention to consume the losslessly compressed KV cache directly. ZipFusion co-designs the compression format and attention kernel to reduce the bytes read from GPU memory during attention. The kernel reconstructs the original values directly in registers using simple bit operations, avoiding a separate decompression pass and easing the memory-bandwidth bound. ZipFusion also reassigns memory freed by compression across layers to store additional KV cache. Its memory manager reserves space for ongoing generation and makes the remaining capacity available to new requests. We implement ZipFusion on top of FlashInfer and integrate it with SGLang. On an NVIDIA H100, ZipFusion achieves about 1.25x speedup over FlashInfer in decode attention. With 8K context, it improves model decode throughput by up to 26.3% over uncompressed SGLang under the same GPU memory budget. At each method's largest supported batch, it also delivers up to 2.76x the throughput of the best-performing lossless KV cache compression baseline.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno).
Categorie arXiv: Hardware Architecture (cs.AR) ; Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2610.33719 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-27T07:56:15.164Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).