Metraipolis › archive › dc_679084459d84

SHAM-Bench: A Span-level Hallucination Benchmark Across Multiple Tasks — arXiv:2610.27370

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.27370
License (as archived)
CC0-1.0
Archived
2026-10-21T07:08:53.853Z
SHA-256 of the archived text
4e1d9863d954690a9f785a1ceef30a4fbb30ea2d511b05d9ea4a18775f209d88
Stable URL of this page
https://metraipolis.com/documento/dc_679084459d84

How to cite

SHAM-Bench: A Span-level Hallucination Benchmark Across Multiple Tasks — arXiv:2610.27370 — arXiv (en), https://arxiv.org/abs/2610.27370 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_679084459d84 (Metraipolis archive dc_679084459d84, sha256 4e1d9863d954690a9f785a1ceef30a4fbb30ea2d511b05d9ea4a18775f209d88)
BibTeX
@misc{dc679084459d84,
  title = {SHAM-Bench: A Span-level Hallucination Benchmark Across Multiple Tasks — arXiv:2610.27370},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.27370},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_679084459d84; sha256 4e1d9863d954690a9f785a1ceef30a4fbb30ea2d511b05d9ea4a18775f209d88},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.27370. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Span-level hallucination detection pinpoints unsupported content in LLM outputs, enabling actionable feedback and targeted mitigation. Existing evaluations for span-level hallucination detection methods are fragmented across tasks, annotation formats, and metrics, and most span-level datasets cover only a single NLG domain, making it difficult to assess whether a detection method actually generalizes across tasks and domains. In this paper, we introduce SHAM-Bench (Span-level Hallucination Benchmark Across Multiple Tasks), a unified benchmark for span-level hallucination detection spanning six NLG tasks, aggregated from existing human-annotated datasets totaling 6,461 examples under a common schema and evaluation protocol. Using SHAM-Bench, we evaluate existing hallucination detection methods and find that fine-tuned detectors often perform well in-domain but degrade under distribution shift, whereas prompt-based methods with reasoning are more robust across tasks. Our analysis further shows that subtle unsupported content remains difficult to detect and span-level evaluation is still challenged by ambiguous span boundaries and noisy labels. We release the dataset, evaluation code, and a public leaderboard to support future research on broadly generalizable hallucination detection.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.27370 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-21T07:05:58.113Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).