Metraipolis › archive › dc_8792cb22ea93

LoopKV: Loop-Aware KV Cache Compression for Serving Looped Transformers — arXiv:2611.04137

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.04137
License (as archived): CC0-1.0
Archived: 2026-11-04T12:33:40.646Z
SHA-256 of the archived text: 549c55d2662ffb650134ca718a44296cd5bd7c75817b697f431357ae0fc83754
Stable URL of this page: https://metraipolis.com/documento/dc_8792cb22ea93

How to cite

LoopKV: Loop-Aware KV Cache Compression for Serving Looped Transformers — arXiv:2611.04137 — arXiv (en), https://arxiv.org/abs/2611.04137 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8792cb22ea93 (Metraipolis archive dc_8792cb22ea93, sha256 549c55d2662ffb650134ca718a44296cd5bd7c75817b697f431357ae0fc83754)

BibTeX

@misc{dc8792cb22ea93,
  title = {LoopKV: Loop-Aware KV Cache Compression for Serving Looped Transformers — arXiv:2611.04137},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.04137},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8792cb22ea93; sha256 549c55d2662ffb650134ca718a44296cd5bd7c75817b697f431357ae0fc83754},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.04137. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Looped transformers apply a shared block several times per token to improve accuracy without adding parameters. For models such as Ouro that keep a separate key-value (KV) cache per loop, cache memory limits the batch size that amortizes repeated weight reads. Uniform-bit quantization and fixed rules per loop do not allocate precision according to each layer's sensitivity in each loop. Measurements on Ouro show that quantization sensitivity varies across (loop, layer) cells, and the sum of individual cell damages approximates the overall output damage. Sharing damage also varies by cell, with later loops generally tolerating sharing better. Skipping low-attention values to save reads produces errors that compound across loops. LoopKV uses a one-time offline profile to choose 4-bit storage, 2-bit storage or reuse of the final loop's KV for each cell. The planner minimizes reads under budgets for predicted damage and storage. During decoding, each loop reads the high 2 bits of each 4-bit value and fetches the low bits only where its own attention weights require 4-bit precision. A single fused kernel in vLLM executes the plan. Serving Ouro-1.4B on one RTX PRO 6000, the memory-optimized LoopKV-M plan stores 35% less KV than uniform 4-bit quantization (KIVI-4) on the same fused kernel and CUDA graph. It sustains 1.7-2.2x KIVI-4's goodput under a latency service-level objective with a 16K-token maximum length. None of its averages on six public benchmarks falls significantly below BF16's.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL); Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2611.04137 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T12:27:02.068Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).