Metraipolis › archive › dc_100fdd218600
Don't `Well, Actually' Me Unless You Know What You're Talking About: Weak Presupposition Verification Degrades General QA Performance — arXiv:2608.06539
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.06539
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T09:46:44.220Z
- SHA-256 of the archived text
7259570c3f412e31edf44f1e0f3ca97b9f2fad763d1abf459fea4f10085c01e6- Stable URL of this page
https://metraipolis.com/documento/dc_100fdd218600
How to cite
Don't `Well, Actually' Me Unless You Know What You're Talking About: Weak Presupposition Verification Degrades General QA Performance — arXiv:2608.06539 — arXiv (en), https://arxiv.org/abs/2608.06539 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_100fdd218600 (Metraipolis archive dc_100fdd218600, sha256 7259570c3f412e31edf44f1e0f3ca97b9f2fad763d1abf459fea4f10085c01e6)
BibTeX
@misc{dc100fdd218600,
title = {Don't `Well, Actually' Me Unless You Know What You're Talking About: Weak Presupposition Verification Degrades General QA Performance — arXiv:2608.06539},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.06539},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_100fdd218600; sha256 7259570c3f412e31edf44f1e0f3ca97b9f2fad763d1abf459fea4f10085c01e6},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.06539. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
False-presupposition QA (FPQA) tests LLMs on their ability to identify false presuppositions in questions and abstain or correct them rather than reinforcing false assumptions. The common approach reduces the task to prompting LLMs to extract presuppositions and fact checking each presupposition. While the performance on dedicated benchmarks keeps improving, evaluation largely focuses on questions with false presuppositions (FPQs) while ignoring the performance on ``normal'' questions (TPQs). Since many benchmarks over-represent FPQs compared to their natural occurrence, the result is that performance on these benchmarks doesn't reflect real-world QA performance. Through extensive experiments across various model families, sizes, and benchmarks, we show that methods that perform better on FPQs tend to perform worse on TPQs. Our analysis reveals this is the result of weak fact checking modules that reject also true presuppositions. We hope our findings will help guide future work toward FPQA methods that generalize well to realistic settings. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-10 in cs.CL). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2608.06539 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T09:46:37.201Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-10?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).