Metraipolis › archive › dc_641a82e8d993

HHR: Hierarchical Hash Retrieval for Efficient LLM Generation — arXiv:2610.01230

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.01230
License (as archived)
CC0-1.0
Archived
2026-10-06T12:04:14.326Z
SHA-256 of the archived text
bdb0e22c69722d8b08d6be3d6795628762b2ea7160f1bf4d051122bb987472c2
Stable URL of this page
https://metraipolis.com/documento/dc_641a82e8d993

How to cite

HHR: Hierarchical Hash Retrieval for Efficient LLM Generation — arXiv:2610.01230 — arXiv (en), https://arxiv.org/abs/2610.01230 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_641a82e8d993 (Metraipolis archive dc_641a82e8d993, sha256 bdb0e22c69722d8b08d6be3d6795628762b2ea7160f1bf4d051122bb987472c2)
BibTeX
@misc{dc641a82e8d993,
  title = {HHR: Hierarchical Hash Retrieval for Efficient LLM Generation — arXiv:2610.01230},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.01230},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_641a82e8d993; sha256 bdb0e22c69722d8b08d6be3d6795628762b2ea7160f1bf4d051122bb987472c2},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.01230. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Efficient long-context inference is essential for large language models (LLMs), yet it poses a severe computational bottleneck. Hash-based retrieval offers an efficient alternative by encoding queries and keys into binary codes and using Hamming distance for key selection. However, this leads to a critical mismatch between Hamming distance and attention relevance. Query-Key logits depend jointly on directional similarity and feature magnitudes, whereas hash binarization discards magnitude information, causing both false-positive retrieval of low-logit keys and false-negative omission of high-logit keys. To address these failures, we propose Hierarchical Hash Retrieval (HHR), a coarse-to-fine framework that progressively improves retrieval accuracy through Geometry-Aware Key Routing (GKR) and Learned Hash Projection (LHP). GKR learns a head-wise orthogonal transformation to redistribute feature magnitudes and derive more discriminative page-level logit bounds, enabling effective pruning of low-logit keys while preserving important candidates. LHP then learns a head-wise projection space that aligns Hamming distance with the true Query-Key relevance ranking for fine-grained retrieval. By combining GKR and LHP, HHR suppresses false positives and recovers false negatives, substantially improving the fidelity of hash-based sparse attention. Extensive experiments across diverse LLMs and benchmarks demonstrate that HHR achieves superior performance over existing methods. For example, on LongBench, HHR improves the average score by 1.10 points and, at a context length of 128K, achieves up to a 3.30x decoding speedup and a 2.83x end-to-end speedup for Llama-3.1-8B-Instruct. The code is publicly available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-02 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.01230 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T10:21:27.925Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).