Metraipolis › archive › dc_6e9531e07e32
Fractional Decay KV-Cache: Ownership-Aware Memory Management for Improved Inference Relevancy in Dialog Systems — arXiv:2608.18098
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.18098
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T13:44:47.805Z
- SHA-256 of the archived text
8e2f8396b3eed067987fec0280bbe6111f2bd22bd929ba78329e6e9c80783e7d- Stable URL of this page
https://metraipolis.com/documento/dc_6e9531e07e32
How to cite
Fractional Decay KV-Cache: Ownership-Aware Memory Management for Improved Inference Relevancy in Dialog Systems — arXiv:2608.18098 — arXiv (en), https://arxiv.org/abs/2608.18098 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6e9531e07e32 (Metraipolis archive dc_6e9531e07e32, sha256 8e2f8396b3eed067987fec0280bbe6111f2bd22bd929ba78329e6e9c80783e7d)
BibTeX
@misc{dc6e9531e07e32,
title = {Fractional Decay KV-Cache: Ownership-Aware Memory Management for Improved Inference Relevancy in Dialog Systems — arXiv:2608.18098},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.18098},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6e9531e07e32; sha256 8e2f8396b3eed067987fec0280bbe6111f2bd22bd929ba78329e6e9c80783e7d},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.18098. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Key-value (KV) caching is essential for efficient autoregressive inference in transformer based dialog systems, yet existing strategies treat all cached entries uniformly or apply coarse eviction heuristics that fail to adapt as dialog topics evolve. We propose Fractional Decay KV-Cache (FD-KVC), a novel algorithm that maintains a dual-channel scoring mechanism for each cached KV pair: a cumulative attention channel that tracks aggregate importance (akin to H2O), and a recency-weighted relevance channel governed by temporal decay and reinforcement-inspired updates. The combination enables FD-KVC to both preserve historically important tokens and rapidly adapt when dialog topics shift. An adaptive learning rate driven by an ownership loss function ensures convergence without oscillation. FD-KVC operates entirely on CPU with negligible overhead. Across five diverse multi-turn dialog scenarios with 600 dialogs each, FD-KVC outperforms H2O, the state-of-the-art heavy-hitter baseline, by +6.7% on composite late-turn alignment, with improvements of +127% on topic-shift, +87% on gradual evolution, and +30% on mixed-topic dialogs. FD-KVC adapts to new topics 3.6X faster than H2O and achieves the highest topic diversity (80.6%) across all methods. Ablation studies confirm the contribution of each component. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-20 in cs.AI). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2608.18098 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T13:41:17.901Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-20?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).