Metraipolis › archive › dc_101d557cb651
ATTUNER: Recomputation-Free KV Cache Reuse via Query-Side Adaptation — arXiv:2609.36722
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.36722
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T16:56:41.161Z
- SHA-256 of the archived text
71126db2e3b5744d89e28788d4b0beaa05118b4c53841ddf9dff9b3d364885c2- Stable URL of this page
https://metraipolis.com/documento/dc_101d557cb651
How to cite
ATTUNER: Recomputation-Free KV Cache Reuse via Query-Side Adaptation — arXiv:2609.36722 — arXiv (en), https://arxiv.org/abs/2609.36722 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_101d557cb651 (Metraipolis archive dc_101d557cb651, sha256 71126db2e3b5744d89e28788d4b0beaa05118b4c53841ddf9dff9b3d364885c2)
BibTeX
@misc{dc101d557cb651,
title = {ATTUNER: Recomputation-Free KV Cache Reuse via Query-Side Adaptation — arXiv:2609.36722},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.36722},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_101d557cb651; sha256 71126db2e3b5744d89e28788d4b0beaa05118b4c53841ddf9dff9b3d364885c2},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.36722. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language model (LLM) agents repeatedly load reusable content, such as skills, documents, and memory entries, into the current context. Re-encoding this content for every request wastes computation. Position-independent caching (PIC) alleviates this by encoding each artifact independently and reusing its key-value (KV) states at arbitrary positions, but it incurs a quality loss relative to full-context prefill. Existing methods repair this loss by restoring global position IDs or recomputing selected tokens. In this work, we isolate the source of the loss, finding that the positional mismatch has minor effect, and independently cached artifacts retain faithful representations: reading a provided artifact stays largely accurate, and performance degrades only when the model must select among multiple artifacts. Moreover, replacing PIC's attention scores with full-prefill scores recovers performance with the cached KV unchanged, localizing the failure to the attention rather than KV recomputation. Motivated by this, we propose \textsc{Attuner}, a query-side adaptation method that learns to read a frozen artifact cache. \textsc{Attuner} inserts low-rank adapters into the query projections and is trained by distilling full-prefill distribution into the student. It trains fewer than 0.05\% of the model parameters and, at inference, requires neither cache recomputation nor a full-context reference. On Qwen3-4B and Qwen3-8B across seven benchmarks covering skills, documents, memory, and code, \textsc{Attuner} substantially outperforms prior PIC baselines in both in-domain and out-of-domain settings, matches full-context prefill quality while providing up to $3.73\times$ speedup.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.36722 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T16:27:03.237Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).