Metraipolis › archive › dc_2086a529ee87
A Universal Context-Reuse Layer for Cross-Model KV Sharing — arXiv:2608.30963
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.30963
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T07:55:10.097Z
- SHA-256 of the archived text
e729032168631b4859390216e2857d6033314d7d49b50bdafcea72abfb5b9f1e- Stable URL of this page
https://metraipolis.com/documento/dc_2086a529ee87
How to cite
A Universal Context-Reuse Layer for Cross-Model KV Sharing — arXiv:2608.30963 — arXiv (en), https://arxiv.org/abs/2608.30963 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2086a529ee87 (Metraipolis archive dc_2086a529ee87, sha256 e729032168631b4859390216e2857d6033314d7d49b50bdafcea72abfb5b9f1e)
BibTeX
@misc{dc2086a529ee87,
title = {A Universal Context-Reuse Layer for Cross-Model KV Sharing — arXiv:2608.30963},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.30963},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2086a529ee87; sha256 e729032168631b4859390216e2857d6033314d7d49b50bdafcea72abfb5b9f1e},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.30963. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Modern large language model (LLM) serving systems increasingly operate over repeated or shared context, yet each model typically performs its own prefill computation even when another model has already processed the same input. Existing KV-cache reuse mechanisms substantially reduce redundant computation within a single model, but generally assume that the producer and consumer of a cache are identical. We study \emph{cross-model KV sharing}, which translates the KV state produced by a source model into a representation that can be consumed by a different target model, including models that differ in scale, architecture, attention configuration, tokenizer, and model family. We evaluate the approach in both within-family and cross-family settings. For Qwen2.5-7B $\rightarrow$ Qwen2.5-1.5B, translated KV states improve LongBench2 accuracy from 27.59\% to 34.48\%, a gain of 6.89 percentage points over the native 1.5B baseline, while reducing handoff cost relative to native target prefill. For the cross-family Qwen2.5-1.5B $\rightarrow$ Gemma-2-2B setting, KV handoff reduces target-side prefill cost by up to 67.05\% at 4K context length while maintaining decoding perplexity close to native-model baselines. In a more heterogeneous Llama3.1-70B $\rightarrow$ Qwen2.5-7B setting, cross-family handoff achieves 44.0\% accuracy compared with 45.7\% for native Qwen2.5-7B inference, while reducing measured latency from 899ms to 138ms. These results provide initial evidence that KV states can serve as transferable computational representations rather than strictly model-local caches, and motivate \emph{context mobility} as a systems abstraction for reducing redundant prefill across heterogeneous LLM and multi-agent inference workflows.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.30963 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T07:28:26.929Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).