Metraipolis › archive › dc_93df8df65154
ReKV: Overlapping Quantized KV Transfer and Suffix Recomputation — arXiv:2611.08770
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.08770
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-09T22:36:29.331Z
- SHA-256 of the archived text
40dcf32abe33ec52b52d6d2e74bf70d121a2d1de975435cd4413b7dafe52047a- Stable URL of this page
https://metraipolis.com/documento/dc_93df8df65154
How to cite
ReKV: Overlapping Quantized KV Transfer and Suffix Recomputation — arXiv:2611.08770 — arXiv (en), https://arxiv.org/abs/2611.08770 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_93df8df65154 (Metraipolis archive dc_93df8df65154, sha256 40dcf32abe33ec52b52d6d2e74bf70d121a2d1de975435cd4413b7dafe52047a)
BibTeX
@misc{dc93df8df65154,
title = {ReKV: Overlapping Quantized KV Transfer and Suffix Recomputation — arXiv:2611.08770},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.08770},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_93df8df65154; sha256 40dcf32abe33ec52b52d6d2e74bf70d121a2d1de975435cd4413b7dafe52047a},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.08770. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Prefill--decode disaggregation moves a prompt's key--value (KV) cache between workers before the first token can be generated. Reducing this handoff to either cache compression or recomputation misses a useful choice: which layers to transmit, at what precision, and which layers to execute again on the decode worker. ReKV transfers a quantized prefix KV cache and one BF16 boundary state, then reconstructs a contiguous suffix while the prefix is in flight. In a model-shaped synthetic benchmark on cross-node H100s, the Q8 configuration meeting separately evaluated quality criteria reduces 8K handoff latency by $2.36\times$, $2.84\times$, and $3.65\times$ over BF16 transfer for Qwen2.5-7B, Qwen2.5-3B, and Qwen3-8B. It is also faster than the best measured transfer-only or recompute-only action at those points. A separate real-weight BF16 recomputation experiment reduces prefill-to-token-return time by $1.16$--$1.63\times$. The preferred action changes with prompt length and link bandwidth. Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno). Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC). Pagina del lavoro: https://arxiv.org/abs/2611.08770 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-09T22:31:17.038Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).