Metraipolis › archive › dc_6b53e6e4eec4
Halluscoring 2026: The first shared task on llms hallucination detection and answer verification — arXiv:2609.38355
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.38355
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T13:13:21.253Z
- SHA-256 of the archived text
e0eb5f8cc18255171ea29b38e730dd150be6057c90bbfcf47105a7ebdb91a068- Stable URL of this page
https://metraipolis.com/documento/dc_6b53e6e4eec4
How to cite
Halluscoring 2026: The first shared task on llms hallucination detection and answer verification — arXiv:2609.38355 — arXiv (en), https://arxiv.org/abs/2609.38355 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6b53e6e4eec4 (Metraipolis archive dc_6b53e6e4eec4, sha256 e0eb5f8cc18255171ea29b38e730dd150be6057c90bbfcf47105a7ebdb91a068)
BibTeX
@misc{dc6b53e6e4eec4,
title = {Halluscoring 2026: The first shared task on llms hallucination detection and answer verification — arXiv:2609.38355},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.38355},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6b53e6e4eec4; sha256 e0eb5f8cc18255171ea29b38e730dd150be6057c90bbfcf47105a7ebdb91a068},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.38355. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
We present HalluScoring 2026, a shared task for evaluating hallucination detection and factual verification in Arabic question answering under challenging generalization settings. Its four subtasks are organized into two tasks. Task~1 evaluates binary hallucination detection, considering generalization to unseen questions (Subtask 1.1) and responses generated by unseen LLMs (Subtask 1.2). Task 2 extends the evaluation beyond detection by requiring the systems to additionally identify the correct factual answer from six related candidates, covering Islamic knowledge (Subtask 2.1) and general knowledge (Subtask 2.2). The shared task is based on two Arabic datasets: HalluScore and HalluTruthQA. A total of 13 teams participated in the shared task, ten of which submitted system description papers. The results of Task 1 demonstrate that hallucination detection remains challenging. On the Task 1 test sets, the top-ranked systems achieved AUC-ROC scores of 0.7717 for Subtask 1.1 (REGLAT) and 0.7670 for Subtask 1.2 (NAMAA). Under assisted evaluation, the highest combined detection and answer-selection scores for Subtasks 2.1 and 2.2 were 0.8824 and 0.8565, respectively. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in cs.CL). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.38355 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T12:44:51.721Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).