Metraipolis › archive › dc_a4dc67d03f28
A Grounded and Decomposed Framework for Relation-Level Hallucination Evaluation in Abstractive Summarization — arXiv:2608.08180
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.08180
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T08:49:55.825Z
- SHA-256 of the archived text
9c239bebe4ec4f625175eb5d1bd0031a4b2c21203f9e0f3f250eae433910b9ba- Stable URL of this page
https://metraipolis.com/documento/dc_a4dc67d03f28
How to cite
A Grounded and Decomposed Framework for Relation-Level Hallucination Evaluation in Abstractive Summarization — arXiv:2608.08180 — arXiv (en), https://arxiv.org/abs/2608.08180 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a4dc67d03f28 (Metraipolis archive dc_a4dc67d03f28, sha256 9c239bebe4ec4f625175eb5d1bd0031a4b2c21203f9e0f3f250eae433910b9ba)
BibTeX
@misc{dca4dc67d03f28,
title = {A Grounded and Decomposed Framework for Relation-Level Hallucination Evaluation in Abstractive Summarization — arXiv:2608.08180},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.08180},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a4dc67d03f28; sha256 9c239bebe4ec4f625175eb5d1bd0031a4b2c21203f9e0f3f250eae433910b9ba},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.08180. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Abstractive text summarization systems frequently generate fluent yet unfaithful summaries by fabricating or distorting relationships between entities and events. Such relation-level hallucinations undermine the reliability of generated summaries, particularly in high-stakes domains. In this work, we present a refined and grounded framework for evaluating relation hallucination in abstractive summarization. We present the empirical Relation Hallucination Index (RHI) by introducing a dependency-aware relation extraction algorithm that incorporates lemmatization-based normalization, named entity grounded subject resolution, passive agent recovery, negation-aware verb modeling, reporting verb filtering, nominal relation fallback, clausal propagation, and systematic deduplication. These enhancements improve the structural fidelity of extracted relation triples and reduce spurious matches during evaluation. In addition, we introduce a normalized formulation of RHI to ensure scale-invariant comparison between datasets and models. The revised metric decomposes hallucination into interpretable components, aggregates relation hallucination metric into a normalized relation faithfulness score. Extensive evaluation across multiple state-of-the-art summarization models demonstrates that the grounded extraction process yields more stable and discriminative hallucination measurements. The proposed framework advances automated relation-level faithfulness evaluation and supports coherence-aware, hallucination-sensitive model analysis. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-11 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2608.08180 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T08:49:41.852Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).