Metraipolis › archive › dc_d8ea9df7aa2b

EarlySelectKV: Anticipatory KV Selection via Cross-Layer Query Proxies — arXiv:2611.01715

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.01715
License (as archived): CC0-1.0
Archived: 2026-11-03T11:10:36.711Z
SHA-256 of the archived text: 209e9fa7d3d33c320b5a66c8986242e2284b75f76a934c596528c85c94c80a33
Stable URL of this page: https://metraipolis.com/documento/dc_d8ea9df7aa2b

How to cite

EarlySelectKV: Anticipatory KV Selection via Cross-Layer Query Proxies — arXiv:2611.01715 — arXiv (en), https://arxiv.org/abs/2611.01715 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d8ea9df7aa2b (Metraipolis archive dc_d8ea9df7aa2b, sha256 209e9fa7d3d33c320b5a66c8986242e2284b75f76a934c596528c85c94c80a33)

BibTeX

@misc{dcd8ea9df7aa2b,
  title = {EarlySelectKV: Anticipatory KV Selection via Cross-Layer Query Proxies — arXiv:2611.01715},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.01715},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d8ea9df7aa2b; sha256 209e9fa7d3d33c320b5a66c8986242e2284b75f76a934c596528c85c94c80a33},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01715. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Query-conditioned sparse KV selection reduces long-context inference cost by attending to a subset of historical KV entries at each decoding step. Existing selectors must wait for the target-layer query before scoring candidates and generating indices, placing routing on the immediate pre-attention path. We introduce EarlySelectKV, a training-free transformation that uses proxy queries from earlier hidden states to precompute selected KV indices while retaining the real target-layer query for final attention. Tests with RocketKV, Quest, and Loki show no consistent pattern of quality degradation across the evaluated settings. In the primary RocketKV evaluation, ES-RocketKV differs by +0.018 points on LongBench, approximately -0.01 points on RULER, and +0.25 percentage points on representative NIAH/Passkey settings. In a TensorRT-LLM deployment with CPU-offloaded routing, ES-RocketKV improves end-to-end output throughput by up to 12.4% and reduces GPU-resident memory by 11.1% of the combined KV/K-summary pool. This makes routing timing an explicit optimization target alongside KV subset selection in sparse long-context decoding. Code is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2611.01715 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:59:52.312Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).