Metraipolis › archive › dc_fcb85d7e047f

Beyond Mean Attention: Diversity-Aware, Layer-Wise Scoring for KV Cache Eviction — arXiv:2609.30738

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.30738
License (as archived): CC0-1.0
Archived: 2026-10-08T02:03:58.684Z
SHA-256 of the archived text: 977e9e2074dafdc0674f2072adcd702c67d6216d1d4c3ae2a3d27fc3a11501e3
Stable URL of this page: https://metraipolis.com/documento/dc_fcb85d7e047f

How to cite

Beyond Mean Attention: Diversity-Aware, Layer-Wise Scoring for KV Cache Eviction — arXiv:2609.30738 — arXiv (en), https://arxiv.org/abs/2609.30738 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fcb85d7e047f (Metraipolis archive dc_fcb85d7e047f, sha256 977e9e2074dafdc0674f2072adcd702c67d6216d1d4c3ae2a3d27fc3a11501e3)

BibTeX

@misc{dcfcb85d7e047f,
  title = {Beyond Mean Attention: Diversity-Aware, Layer-Wise Scoring for KV Cache Eviction — arXiv:2609.30738},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.30738},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fcb85d7e047f; sha256 977e9e2074dafdc0674f2072adcd702c67d6216d1d4c3ae2a3d27fc3a11501e3},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.30738. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

KV cache eviction methods such as SnapKV and PyramidKV rank tokens solely by mean attention over a small observation window. We study a unified score, $\mu_i+\lambda_1\sigma_i+\lambda_2\mathrm{corr}(i,S)$, adding attention dispersion across window queries and redundancy relative to selected tokens. For $\lambda_2<0$, the score penalizes similarity to selected tokens as in maximal marginal relevance (MMR), without extra forward passes. To test whether this relevance-diversity balance should vary with depth, we compare fixed global coefficients with three-segment and quadratic profiles. Only these depth profiles are searched on a development split under a $\sinh$ reparameterization. On all 16 English LongBench datasets with Mistral-7B at a budget of 64 entries per layer, a single global diversification constant improves 13 of 16 datasets (macro +1.1); the gain holds at budget 32 and narrows at 128. Per-dataset search finds no detectable layer structure on most datasets; on passage retrieval it finds a large one: a mid-layer sign flip that rewards similarity and is worth +9.6 over the baseline at budget 64 and, without re-tuning, +13.2 over the global constant at budget 128. Ablations attribute the gain to the redundancy term; replaying every accepted search state on the held-out test set separates genuine structure from tuning noise.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-28 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.30738 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T01:38:49.436Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).