Metraipolis › archive › dc_bcb84e94245e
Corpus-Supported Distractors: Auditing Retrieval-Augmented Quiz Generation Against the Stem (2026) · DOI 10.20944/preprints202609.2167.v1
- Source
- Metraipolis archive
- Original
- https://doi.org/10.20944/preprints202609.2167.v1
- License (as archived)
- CC-BY-4.0
- Archived
- 2026-10-07T09:09:58.306Z
- SHA-256 of the archived text
f0654ae76fa2fc50ec217bd01fb8276db9b607f6d56b49cc67379634d4eceae8- Stable URL of this page
https://metraipolis.com/documento/dc_bcb84e94245e
How to cite
Corpus-Supported Distractors: Auditing Retrieval-Augmented Quiz Generation Against the Stem (2026) · DOI 10.20944/preprints202609.2167.v1 — Metraipolis archive, https://doi.org/10.20944/preprints202609.2167.v1 (CC-BY-4.0); archived copy: https://metraipolis.com/documento/dc_bcb84e94245e (Metraipolis archive dc_bcb84e94245e, sha256 f0654ae76fa2fc50ec217bd01fb8276db9b607f6d56b49cc67379634d4eceae8)
BibTeX
@misc{dcbcb84e94245e,
title = {Corpus-Supported Distractors: Auditing Retrieval-Augmented Quiz Generation Against the Stem (2026) · DOI 10.20944/preprints202609.2167.v1},
howpublished = {Metraipolis archive},
url = {https://doi.org/10.20944/preprints202609.2167.v1},
note = {License CC-BY-4.0; archived copy https://metraipolis.com/documento/dc_bcb84e94245e; sha256 f0654ae76fa2fc50ec217bd01fb8276db9b607f6d56b49cc67379634d4eceae8},
year = {2026}
}Archived text
Reproduced under CC-BY-4.0. Attribution: Metraipolis archive, https://doi.org/10.20944/preprints202609.2167.v1. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Articolo recente con licenza aperta: scheda dai metadati Crossref (posted-content). Titolo: Corpus-Supported Distractors: Auditing Retrieval-Augmented Quiz Generation Against the Stem Autori: Mohamed Farag Pubblicato: 2026-09-24 · editore: MDPI AG DOI: https://doi.org/10.20944/preprints202609.2167.v1 Licenza dell'articolo: CC-BY-4.0 Abstract (testo degli autori, CC-BY-4.0): A multiple-choice item whose wrong option is also a correct answer to its own stem is not a hard item, it is a broken one, and every faithfulness metric a retrieval-augmented quiz generator currently ships will score that option as clean. The reason is structural. Groundedness is a two-place relation between an option and the retrieved passage, item validity is a three-place relation among stem, option and passage, and no deployed metric supplies the third argument. We name the failure a corpus-supported distractor, build a stem-conditioned detector for it, and run that detector inside a ten-arm design on a 13,461-passage science-exam corpus standing in for course material, with three seeds and 100 items per arm per seed. Grounded generation leaves the corpus supporting a wrong option on 3.24% of distractor slots, against 0.68% for a generator denied the passage, and 1.00% of slots (9 of 894, Wilson 95% CI [0.34%, 1.79%]) survive the further test that the option stays supported under a narrowed stem, against 0.11% (1 of 885) for the ungrounded arm. Three seeds cannot certify that gap (paired t(2) = -1.84, p = 0.21) and a replication on a second corpus does not reproduce it at all. Four grounding-based faithfulness filters each raise the conditional rate to between 1.34% and 2.01%, which looks like the filter making the defect worse until a retention-matched random-drop null reaches 1.79% by discarding items at random. Counted rather than divided, the three entailment-family filters left the flagged-slot count at 9 while discarding half the pool, and they are indistinguishable from the attrition null on the counts themselves (Fisher exact p = 1.00). The rate rose because the denominator halved. We report the arithmetic, the pool size at which the statistical question becomes answerable, and a circularity that no pool size fixes, namely that a filter selecting on entailment cannot be scored by an endpoint that reads the same entailment signal. Fonte: https://api.crossref.org/works/10.20944%2Fpreprints202609.2167.v1 — metadati Crossref, consultati il 2026-10-06. Interrogazione: «retrieval augmented generation».