Metraipolis › archive › dc_87e448458987

SAGE: SLO-Aware Adaptive Retrieval for Production RAG Systems — arXiv:2608.08237

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.08237
License (as archived)
CC0-1.0
Archived
2026-10-19T08:50:45.562Z
SHA-256 of the archived text
de01403b10a0ad58cd0202240cbf675982d2067ffe45d837021a49bcabc0aef3
Stable URL of this page
https://metraipolis.com/documento/dc_87e448458987

How to cite

SAGE: SLO-Aware Adaptive Retrieval for Production RAG Systems — arXiv:2608.08237 — arXiv (en), https://arxiv.org/abs/2608.08237 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_87e448458987 (Metraipolis archive dc_87e448458987, sha256 de01403b10a0ad58cd0202240cbf675982d2067ffe45d837021a49bcabc0aef3)
BibTeX
@misc{dc87e448458987,
  title = {SAGE: SLO-Aware Adaptive Retrieval for Production RAG Systems — arXiv:2608.08237},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.08237},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_87e448458987; sha256 de01403b10a0ad58cd0202240cbf675982d2067ffe45d837021a49bcabc0aef3},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.08237. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Retrieval-Augmented Generation (RAG) systems in production operate under strict service level objectives (SLOs) on tail latency and infrastructure cost. However, standard retrieval pipelines rely on fixed retrieval budgets that ignore query difficulty, over-retrieving for easy queries and under-serving hard ones, forcing operators to trade answer quality against SLO compliance. This paper proposes SAGE, a learned SLO-aware adaptive retrieval policy that dynamically selects the number of passages k per query. SAGE uses lightweight features derived from initial retrieval (e.g., score distributions, rank gaps, lexical signals) and is trained offline via imitation learning from an oracle that approximates optimal latency-quality trade-offs. At inference, it adds no LLM calls and minimal overhead. On Natural Questions, under a 5s P95 latency SLO, SAGE achieves 95% SLO compliance versus 30% for the best static baseline (k=20), reduces P95 latency by 36% and retrieval cost by 51% with only 2 percentage points Exact Match (EM) loss. A single policy trained on Natural Questions generalizes across HotpotQA, UnSeenTimeQA, and four LLM families (Llama, Qwen, Mistral, Gemma), consistently yielding +45-52 point SLO improvements without quality degradation.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-11 in cs.CL).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL); Distributed, Parallel, and Cluster Computing (cs.DC); Information Retrieval (cs.IR).
Pagina del lavoro: https://arxiv.org/abs/2608.08237 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T08:49:41.976Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).