Metraipolis › archive › dc_fc3bbd3bbb03

LinearKV: One Cached State Suffices for Position-Independent Caching in Hybrid LLMs — arXiv:2608.11231

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.11231
License (as archived): CC0-1.0
Archived: 2026-10-19T05:15:12.186Z
SHA-256 of the archived text: 47dde5d1c7a0e68c4be9d799c79b485b97aec1dfac01c84ce5b1ebb7c6f4556c
Stable URL of this page: https://metraipolis.com/documento/dc_fc3bbd3bbb03

How to cite

LinearKV: One Cached State Suffices for Position-Independent Caching in Hybrid LLMs — arXiv:2608.11231 — arXiv (en), https://arxiv.org/abs/2608.11231 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fc3bbd3bbb03 (Metraipolis archive dc_fc3bbd3bbb03, sha256 47dde5d1c7a0e68c4be9d799c79b485b97aec1dfac01c84ce5b1ebb7c6f4556c)

BibTeX

@misc{dcfc3bbd3bbb03,
  title = {LinearKV: One Cached State Suffices for Position-Independent Caching in Hybrid LLMs — arXiv:2608.11231},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.11231},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fc3bbd3bbb03; sha256 47dde5d1c7a0e68c4be9d799c79b485b97aec1dfac01c84ce5b1ebb7c6f4556c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.11231. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

LLM serving is increasingly accelerated by position-independent caching (PIC). Existing PIC methods, however, are built for full-attention models, where a token-indexed KV cache underlies its core operations: matching reusable token chunks, concatenating their KV entries, and selectively recomputing a few tokens to restore cross-chunk context. Hybrid LLMs break these primitives---they replace most attention layers with linear recurrences that expose only a fixed-size state, leaving no token-indexed KV to concatenate or to locally repair. This raises a natural question: can PIC benefit hybrid models, and what would it take? We present LinearKV, a training-free hybrid-PIC framework. Its key insight is a \emph{decoupled initialization}: each linear layer maps its $K$ matched local states to a single initial state, while full-attention layers concatenate their KV as before. LinearKV is therefore compatible with existing PIC methods, reusing their token selection and recomputation as-is. Under this framework, we find that a \emph{single cached state} suffices as the linear layer's initializer. The algebraically principled alternative---composing all $K$ cached states into the exact full-prefix state, as concurrent work HYPIC does---is unnecessary and, on some architectures, even harmful. We compare the two across three hybrid models and three PIC selectors. On the two GDN models the two tie, both recovering most of full quality (up to $92\%$); on the Mamba-2 model, exact composition instead collapses under every selector---under EPIC, for instance, it recovers only $46.6\%$ of full quality, versus $86.8\%$ for a single cached block initializer. A single state initializer is also cheaper, cutting time-to-first-token to $0.46\times$ full prefill versus a further $5$--$17\%$ overhead for exact composition; results hold across LongBench QA and RULER at 8K--32K.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-13 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.11231 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T05:08:46.605Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-13?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).