Metraipolis › archive › dc_b652ffa90bbf

KV-Rescue: Recovering Reasoning Language Model KV Eviction Loss via Stepwise Interleaving — arXiv:2608.15797

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.15797
License (as archived): CC0-1.0
Archived: 2026-10-18T17:28:52.983Z
SHA-256 of the archived text: 24ab9b3d105946efa7d50fe458a740ecbdf53bb7b2fe5aa77e97b59d496c1ce5
Stable URL of this page: https://metraipolis.com/documento/dc_b652ffa90bbf

How to cite

KV-Rescue: Recovering Reasoning Language Model KV Eviction Loss via Stepwise Interleaving — arXiv:2608.15797 — arXiv (en), https://arxiv.org/abs/2608.15797 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b652ffa90bbf (Metraipolis archive dc_b652ffa90bbf, sha256 24ab9b3d105946efa7d50fe458a740ecbdf53bb7b2fe5aa77e97b59d496c1ce5)

BibTeX

@misc{dcb652ffa90bbf,
  title = {KV-Rescue: Recovering Reasoning Language Model KV Eviction Loss via Stepwise Interleaving — arXiv:2608.15797},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.15797},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b652ffa90bbf; sha256 24ab9b3d105946efa7d50fe458a740ecbdf53bb7b2fe5aa77e97b59d496c1ce5},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.15797. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

KV-cache eviction caps the memory cost of long reasoning traces but is inherently lossy because the model decodes from a partial view of its history. Under aggressive budgets, this not only lowers accuracy but can also cause runaway degeneration, where the model produces incoherent or repetitive tokens until reaching the length limit. We characterize much of this loss as an information gapf caused by missing context, rather than a capability gap caused by limited model capacity. An evicted 7B model and a full-context 1.5B model make complementary errors, and an oracle choice between their answers recovers 79% of the accuracy gap to the full-KV 7B model. Based on this observation, we propose KV-Rescue, a training-free inference framework that bridges the information gap introduced by KV eviction using a lightweight full-context helper. KV-Rescue interleaves reasoning steps from the two models into a shared trajectory. An online detector uses entropy and compressibility to terminate the generation of incoherent or repetitive base-model candidates early. Across five math benchmarks with Qwen2.5-Math 7B and 72B, KV-Rescue recovers an average of 87% of the accuracy lost to eviction at eviction budget B=64. A decode-cost analysis further shows that preventing runaway degeneration cuts base-model token generation by 43% on average.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.15797 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T16:29:13.830Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).