Metraipolis › archive › dc_ccbf032ec1e6

QRAE: Query-Relevant Attention Entropy for Context Compression in DeepSearch Agents — arXiv:2611.07976

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.07976
License (as archived): CC0-1.0
Archived: 2026-11-06T14:50:48.289Z
SHA-256 of the archived text: d5758ac25a2b9dae92bdd50428bfee395acdadb9b0dc9327113b0f003e586f47
Stable URL of this page: https://metraipolis.com/documento/dc_ccbf032ec1e6

How to cite

QRAE: Query-Relevant Attention Entropy for Context Compression in DeepSearch Agents — arXiv:2611.07976 — arXiv (en), https://arxiv.org/abs/2611.07976 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ccbf032ec1e6 (Metraipolis archive dc_ccbf032ec1e6, sha256 d5758ac25a2b9dae92bdd50428bfee395acdadb9b0dc9327113b0f003e586f47)

BibTeX

@misc{dcccbf032ec1e6,
  title = {QRAE: Query-Relevant Attention Entropy for Context Compression in DeepSearch Agents — arXiv:2611.07976},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.07976},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ccbf032ec1e6; sha256 d5758ac25a2b9dae92bdd50428bfee395acdadb9b0dc9327113b0f003e586f47},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.07976. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

DeepSearch LLM agents repeatedly search and carry retrieved sources into subsequent reasoning steps. This expanding context increases token consumption and latency, although many results do not contribute to later reasoning. We propose Query-Relevant Attention Entropy (QRAE), an online context compression method that uses sparse evidence-assessment heads to decide which intact retrieved chunks should persist across reasoning steps. It scores bounded candidate batches through paired prefills: a query-free control suppresses shared attention responses, while normalized entropy measures the breadth of the positive response within each chunk. High-scoring chunks enter a persistent evidence whitelist and are included in the reasoning context. Uncertain chunks remain outside the reasoning context but are retained in a candidate cache, where they can be retrieved for later reevaluation. QRAE requires neither an additional generative compression pass nor modification of the attention operator. Across multiple DeepSearch benchmarks and reasoning models, QRAE improves task success by an average of 2.35 percentage points and reduces end-to-end latency and reported token usage by an average of 30% and 45%, respectively. These benefits generalize to a model with nearly 100B parameters.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.07976 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-06T14:21:42.639Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).