Metraipolis › archive › dc_ca9f371829af
Pallas: A Proactive KV Cache Migration Framework for LLM Inference in AI-RAN — arXiv:2608.16477
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.16477
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T19:41:39.590Z
- SHA-256 of the archived text
780ee9ff587204fee2b924d37870f0bbdd60a838e492079ae0aac83b76da620a- Stable URL of this page
https://metraipolis.com/documento/dc_ca9f371829af
How to cite
Pallas: A Proactive KV Cache Migration Framework for LLM Inference in AI-RAN — arXiv:2608.16477 — arXiv (en), https://arxiv.org/abs/2608.16477 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ca9f371829af (Metraipolis archive dc_ca9f371829af, sha256 780ee9ff587204fee2b924d37870f0bbdd60a838e492079ae0aac83b76da620a)
BibTeX
@misc{dcca9f371829af,
title = {Pallas: A Proactive KV Cache Migration Framework for LLM Inference in AI-RAN — arXiv:2608.16477},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.16477},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ca9f371829af; sha256 780ee9ff587204fee2b924d37870f0bbdd60a838e492079ae0aac83b76da620a},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.16477. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
AI-RAN brings large language model (LLM) serving close to mobile users, but cellular handover can separate an active request from its inference state: the user attaches to a target base station (gNB) while the large and growing key-value (KV) cache remains at the source. Retaining inference at the source preserves service continuity but persistently increases inter-token latency (ITL), whereas recovering the state at the target restores serving locality but requires KV-cache transfer, recomputation, or a combination of both only after handover, directly prolonging service interruption time (SIT). This work presents Pallas, a \textit{proactive} KV-cache migration framework that prepares the inference state at the predicted target before handover, in parallel with ongoing source-side inference and token delivery. At the preparation trigger, Pallas partitions the token sequence into a stable historical prefix and an evolving suffix. The target reconstructs the prefix through local prefill, while the source streams the KV blocks generated for the suffix. At handover, the target assembles both portions into an up-to-date KV cache and resumes decoding locally, leaving only unfinished preparation to contribute to SIT. An online scheduler selects the \textit{prefetching window}, which determines how early preparation begins before handover, based on mobility predictions and runtime telemetry. Across three LLMs and $100$--$500~\mathrm{Mbps}$ inter-gNB links, our vLLM-based prototype reduces average SIT by factors of $2.28$--$89.68$ over target-side recovery approaches and lowers average ITL by $16.0\%$--$50.0\%$ compared with source-side forwarding.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.16477 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T19:02:44.178Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).