Metraipolis › archive › dc_7ab769bcbe2e

CacheWeave: Learning to Recompute for Efficient Position-Independent KV Caching — arXiv:2611.01735

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.01735
License (as archived): CC0-1.0
Archived: 2026-11-03T11:12:38.678Z
SHA-256 of the archived text: a1d8342e3cdebae3bea5ec3ed19e5775c752d02a7221c9f2d86dfdf10a979919
Stable URL of this page: https://metraipolis.com/documento/dc_7ab769bcbe2e

How to cite

CacheWeave: Learning to Recompute for Efficient Position-Independent KV Caching — arXiv:2611.01735 — arXiv (en), https://arxiv.org/abs/2611.01735 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7ab769bcbe2e (Metraipolis archive dc_7ab769bcbe2e, sha256 a1d8342e3cdebae3bea5ec3ed19e5775c752d02a7221c9f2d86dfdf10a979919)

BibTeX

@misc{dc7ab769bcbe2e,
  title = {CacheWeave: Learning to Recompute for Efficient Position-Independent KV Caching — arXiv:2611.01735},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.01735},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7ab769bcbe2e; sha256 a1d8342e3cdebae3bea5ec3ed19e5775c752d02a7221c9f2d86dfdf10a979919},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01735. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Position-independent caching reuses chunk KV caches across positions and requests. Independent encoding, however, omits cross-chunk interactions and can degrade accuracy. In our study, neither PIC training nor rule-based recomputation, alone or combined, consistently recovers full-prefill accuracy. Oracle-guided recomputation on PIC-trained models improves accuracy at the same budget, motivating better selection. We present CacheWeave, which adapts the full backbone to PIC, then freezes it and trains a model-native block selector on the adapted model's oracles. The selector uses compact cached chunk features and question hidden states without extracting backbone attention scores. It selects contiguous token blocks aligned with KV blocks to support efficient recomputation and limit copy-on-write amplification. Its runtime, CacheWeave-Serve, supports private and shared KV caches, keeps selection and recomputation on the GPU, and stages sliding-window KV for hybrid attention models. Across three models and four tasks, CacheWeave outperforms CacheBlend and ProphetKV by 7.2 and 3.5 percentage points on average at a 5% recomputation budget, with a mean relative accuracy loss of 1.6% to full prefill. With 32-token blocks, CacheWeave-Serve cuts single-request TTFT by up to 16.1x over full prefill and 1.7x over ProphetKV, and raises SLO goodput under load by up to 7.8x.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.01735 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T11:00:07.150Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).