Metraipolis › archive › dc_bf8ccac26296
Evaluating Evidence Sensitivity and Robustness of Hallucination Detectors in Retrieval- Augmented Generation (2026) · DOI 10.21203/rs.3.rs-11400108/v1
- Source
- Metraipolis archive
- Original
- https://doi.org/10.21203/rs.3.rs-11400108/v1
- License (as archived)
- CC-BY-4.0
- Archived
- 2026-10-26T19:00:12.205Z
- SHA-256 of the archived text
348775d169537d43c2977818d9f538c224824f50677a8aa3f78c0d100ea53e6f- Stable URL of this page
https://metraipolis.com/documento/dc_bf8ccac26296
How to cite
Evaluating Evidence Sensitivity and Robustness of Hallucination Detectors in Retrieval- Augmented Generation (2026) · DOI 10.21203/rs.3.rs-11400108/v1 — Metraipolis archive, https://doi.org/10.21203/rs.3.rs-11400108/v1 (CC-BY-4.0); archived copy: https://metraipolis.com/documento/dc_bf8ccac26296 (Metraipolis archive dc_bf8ccac26296, sha256 348775d169537d43c2977818d9f538c224824f50677a8aa3f78c0d100ea53e6f)
BibTeX
@misc{dcbf8ccac26296,
title = {Evaluating Evidence Sensitivity and Robustness of Hallucination Detectors in Retrieval- Augmented Generation (2026) · DOI 10.21203/rs.3.rs-11400108/v1},
howpublished = {Metraipolis archive},
url = {https://doi.org/10.21203/rs.3.rs-11400108/v1},
note = {License CC-BY-4.0; archived copy https://metraipolis.com/documento/dc_bf8ccac26296; sha256 348775d169537d43c2977818d9f538c224824f50677a8aa3f78c0d100ea53e6f},
year = {2026}
}Archived text
Reproduced under CC-BY-4.0. Attribution: Metraipolis archive, https://doi.org/10.21203/rs.3.rs-11400108/v1. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Articolo recente con licenza aperta: scheda dai metadati Crossref (posted-content). Titolo: Evaluating Evidence Sensitivity and Robustness of Hallucination Detectors in Retrieval- Augmented Generation Autori: Ramana Kulanthaivelu Pubblicato: 2026-10-26 · editore: Springer Science and Business Media LLC DOI: https://doi.org/10.21203/rs.3.rs-11400108/v1 Licenza dell'articolo: CC-BY-4.0 Abstract (testo degli autori, CC-BY-4.0): Abstract Retrieval-Augmented Generation (RAG) improves the accessibility of external knowledge for large language models, but generated responses can remain unsupported, contradictory, or otherwise hallucinatory. Conventional hallucination-detector evaluation largely emphasizes aggregate benchmark performance, which may not reveal whether detector decisions remain reliable when the evidence distribution changes. This study presents a controlled evaluation of hallucination detectors from an evidence-sensitivity and robustness perspective. Three detector configurations - TF-IDF with Logistic Regression, TF-IDF with Linear Support Vector Machine, and DeBERTa-v3-base were evaluated using RAGTruth, HaluBench, and TRIVIA+. In-domain evaluation on RAGTruth showed that DeBERTa achieved an AUROC of 0.8483 and Macro-F1 of 0.7616. However, performance declined substantially under distribution shift, with AUROC decreasing to 0.5348 on HaluBench and 0.5910 on TRIVIA+. Controlled evidence interventions showed substantial detector responsiveness, with mean Evidence Sensitivity Index values of 0.2641 for evidence removal and 0.3268 for cross-example evidence substitution. A context-complexity stress test further showed progressive degradation as irrelevant evidence increased from 0% to 100% within a fixed 512-token observation window, with AUROC decreasing from 0.8306 to 0.7325 and Expected Calibration Error increasing from 0.1375 to 0.4072. Input ablation demonstrated that both generated answers and retrieved evidence contribute to detection performance. These findings show that benchmark accuracy alone is insufficient to characterize hallucination-detector reliability and motivate evaluation frameworks that jointly assess generalization, evidence sensitivity, contextual robustness, and calibration. Fonte: https://api.crossref.org/works/10.21203%2Frs.3.rs-11400108%2Fv1 — metadati Crossref, consultati il 2026-10-26. Interrogazione: «retrieval augmented generation».