Metraipolis › archive › dc_85075fccfdd2

CacheRepair: Learning to Repair Cross-Chunk Context in RAG for KV Cache Fusion — arXiv:2609.35139

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.35139
License (as archived)
CC0-1.0
Archived
2026-10-07T22:38:52.205Z
SHA-256 of the archived text
ab45c87a61e63f6753b1c7911f2e265362b8a4c7cbb54d8e7024484f5c352285
Stable URL of this page
https://metraipolis.com/documento/dc_85075fccfdd2

How to cite

CacheRepair: Learning to Repair Cross-Chunk Context in RAG for KV Cache Fusion — arXiv:2609.35139 — arXiv (en), https://arxiv.org/abs/2609.35139 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_85075fccfdd2 (Metraipolis archive dc_85075fccfdd2, sha256 ab45c87a61e63f6753b1c7911f2e265362b8a4c7cbb54d8e7024484f5c352285)
BibTeX
@misc{dc85075fccfdd2,
  title = {CacheRepair: Learning to Repair Cross-Chunk Context in RAG for KV Cache Fusion — arXiv:2609.35139},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.35139},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_85075fccfdd2; sha256 ab45c87a61e63f6753b1c7911f2e265362b8a4c7cbb54d8e7024484f5c352285},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.35139. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multi-document retrieval-augmented generation (RAG) requires a language model to process multiple retrieved text chunks before answering a question. Precomputing each chunk's KV cache independently and concatenating the caches when the chunks are retrieved can accelerate this step. However, the assembled cache lacks cross-chunk attention information, reducing answer quality. Selective recomputation methods recover the missing cross-chunk context by rerunning the target LLM on selected tokens, incurring substantial online computation. We introduce CacheRepair, a lightweight network that learns the difference between independently computed KV caches and those produced by processing the chunks together. The network combines compressed KV features with token embeddings and uses attention that is bidirectional within each chunk and flows from earlier to later chunks. Each repair block receives the compressed cache features, and the predicted residual is added to every document token's cache. Each repair network is trained for a specific frozen target LLM on a generic retrieval corpus and reused across downstream datasets. Our analysis shows that repair reduces KV errors both near chunk boundaries and throughout chunk interiors. Evaluation across three target LLMs and four downstream datasets places CacheRepair on the measured answer-quality-latency Pareto frontier in eleven of twelve model-dataset combinations. Reported time to first token (TTFT) includes online cache transfer and repair. Across all twelve combinations, the largest repairers achieve 1.69-4.61$\times$ speedups in median TTFT over full prefill and improve mean F1 by 2.1-26.1 percentage points over direct cache reuse.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.35139 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T22:14:28.370Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).