Metraipolis › archive › dc_5679ff80734b

Beyond Compression Ratio: High-Throughput KV Compression for Disaggregated LLM Serving — arXiv:2611.04382

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.04382
License (as archived)
CC0-1.0
Archived
2026-11-04T12:34:46.889Z
SHA-256 of the archived text
697a74ce434e0e6df3aa38ea9c6721df15ad1f44e5e238944401a352f988cc4d
Stable URL of this page
https://metraipolis.com/documento/dc_5679ff80734b

How to cite

Beyond Compression Ratio: High-Throughput KV Compression for Disaggregated LLM Serving — arXiv:2611.04382 — arXiv (en), https://arxiv.org/abs/2611.04382 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5679ff80734b (Metraipolis archive dc_5679ff80734b, sha256 697a74ce434e0e6df3aa38ea9c6721df15ad1f44e5e238944401a352f988cc4d)
BibTeX
@misc{dc5679ff80734b,
  title = {Beyond Compression Ratio: High-Throughput KV Compression for Disaggregated LLM Serving — arXiv:2611.04382},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.04382},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5679ff80734b; sha256 697a74ce434e0e6df3aa38ea9c6721df15ad1f44e5e238944401a352f988cc4d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.04382. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

KV cache compression reduces the communication volume of disaggregated large language model serving, but its practical benefit depends not only on compression ratio and model quality, but also on how efficiently the compression pipeline executes. This requirement becomes increasingly important as modern serving systems adopt high-bandwidth interconnects, where inefficient compression can offset a substantial portion of the saved communication time. This paper presents \system, a high-throughput KV cache compression data path built on top of \kvserve. \system improves compression execution in two complementary ways. First, it fuses the distribution transformation and low-bit quantization stages into a KV-aware \tilelang kernel, eliminating intermediate tensor materialization and redundant memory accesses. Second, it replaces the original lossless compression backend with an open, extensible, and searchable backend based on \lc, enabling high-performance compression operator pipelines. The fused kernel achieves up to $5.9\times$ speedup, while the new lossless backend increases compression throughput from below 90~GB/s to over 500~GB/s. Across the evaluated workloads, \system reduces job completion time by up to $2.04\times$. End-to-end evaluation shows that these optimizations allow KV compression to retain meaningful communication benefits under modern high-bandwidth network.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2611.04382 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T12:27:16.999Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).