Metraipolis › archive › dc_dc6be3e755c0

RAG-NIMBF: A Non-Invasive, Modular Benchmarking Framework for Automated RAG Configuration Analysis — arXiv:2610.27816

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.27816
License (as archived)
CC0-1.0
Archived
2026-10-21T07:08:05.298Z
SHA-256 of the archived text
b4acedcb892062c54d831b37102ab6b2ed878abc94691d8a1cdbdd04ecf3e1f0
Stable URL of this page
https://metraipolis.com/documento/dc_dc6be3e755c0

How to cite

RAG-NIMBF: A Non-Invasive, Modular Benchmarking Framework for Automated RAG Configuration Analysis — arXiv:2610.27816 — arXiv (en), https://arxiv.org/abs/2610.27816 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dc6be3e755c0 (Metraipolis archive dc_dc6be3e755c0, sha256 b4acedcb892062c54d831b37102ab6b2ed878abc94691d8a1cdbdd04ecf3e1f0)
BibTeX
@misc{dcdc6be3e755c0,
  title = {RAG-NIMBF: A Non-Invasive, Modular Benchmarking Framework for Automated RAG Configuration Analysis — arXiv:2610.27816},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.27816},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dc6be3e755c0; sha256 b4acedcb892062c54d831b37102ab6b2ed878abc94691d8a1cdbdd04ecf3e1f0},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.27816. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Retrieval-Augmented Generation (RAG) has become the dominant paradigm for grounding the outputs of large language models (LLM) in external knowledge. However, practitioners are faced with a vast configuration space comprising chunking, embedding, retrieval, reranking, prompting and generation models, and there is limited guidance on how these components interact. We present RAG-NIMBF, a modular benchmarking framework that attaches to existing RAG pipelines as a non-invasive evaluation layer. It evaluates the configuration space through controlled ablation with end-to-end experiment tracking. The evaluation combines LLM-based answer quality scores, retrieval correctness measures, and runtime indicators. This study validates the framework across multiple generation models on the SQuAD dataset. This process reveals non-obvious interactions and demonstrates how the experiments are tracked. The findings indicate that the retrieval balance of relevance and diversity substantially enhances the relevance and coverage of retrieved context in comparison to a conventional similarity search, with MMR raising nDCG@5 by +4 to +11 percentage points across the swept models. More detailed prompt instructions meaningfully increase how faithfully generated answers stick to the retrieved evidence, especially for capable and reasoning models (+10 to +13pp on Qwen3.6-27B, GPT-4o-mini and Qwen3.5-think, but negligible on the 20B generator). Overall, RAG-NIMBF transforms the configuration space of RAG into reproducible, comparable experiments. By attaching to existing production pipelines in a non-invasive way, it enables practitioners to systematically evaluate and improve deployed systems without disrupting existing workflows. This replaces ad hoc choices with evidence-based decisions.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Software Engineering (cs.SE) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.27816 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-21T07:05:43.601Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).