Metraipolis › archive › dc_670c3bbb6851

EnterpriseRAG: Benchmarking LLM Instruction Adherence and Robustness under Non-Ideal Enterprise Retrieval — arXiv:2608.11584

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.11584
License (as archived)
CC0-1.0
Archived
2026-10-19T05:15:23.995Z
SHA-256 of the archived text
0c22f192b0ca2321844e747b1f6ebffa05c9b3fe61705f3ad464352e1c646073
Stable URL of this page
https://metraipolis.com/documento/dc_670c3bbb6851

How to cite

EnterpriseRAG: Benchmarking LLM Instruction Adherence and Robustness under Non-Ideal Enterprise Retrieval — arXiv:2608.11584 — arXiv (en), https://arxiv.org/abs/2608.11584 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_670c3bbb6851 (Metraipolis archive dc_670c3bbb6851, sha256 0c22f192b0ca2321844e747b1f6ebffa05c9b3fe61705f3ad464352e1c646073)
BibTeX
@misc{dc670c3bbb6851,
  title = {EnterpriseRAG: Benchmarking LLM Instruction Adherence and Robustness under Non-Ideal Enterprise Retrieval — arXiv:2608.11584},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.11584},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_670c3bbb6851; sha256 0c22f192b0ca2321844e747b1f6ebffa05c9b3fe61705f3ad464352e1c646073},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.11584. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Enterprise RAG deployments face a critical reliability gap: while LLMs satisfy 80% of individual constraints, only 26.8% of responses meet all requirements simultaneously, revealing a 57-point orchestration gap. Existing benchmarks assume clean retrieval with simple queries, failing to capture production conditions where noisy documents and multi-dimensional constraints coexist. We introduce EnterpriseRAG, a benchmark of 983 expert-validated samples across six domains that systematically simulates three failure modes absent from prior work: retrieval noise, knowledge gaps, and factual conflicts, coupled with complex instructions. Evaluation of 13 state-of-the-art LLMs reveals a severe instruction adherence collapse, where high per-constraint satisfaction masks low holistic compliance. Critical findings expose deep barriers under knowledge gaps and factual conflicts, even with reasoning-enhanced inference, indicating production RAG requires explicit context-aware protocols and calibrated judgment. EnterpriseRAG provides a reproducible foundation for measuring and closing these gaps, directly informing deployment decisions for enterprise-scale RAG systems. We will release the benchmark and evaluation framework upon publication.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-13 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.11584 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T05:08:46.631Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-13?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).