Metraipolis › archive › dc_3db7c812ad85

When Repository Labels Are Not Image-Level Truth: A Supervision Auditing Framework for Chest Radiograph AI — arXiv:2608.10084

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.10084
License (as archived)
CC0-1.0
Archived
2026-10-19T06:30:48.425Z
SHA-256 of the archived text
63c27bf258fba841ca086057801fd7313c562d5ce823dae11fd1018e69485e31
Stable URL of this page
https://metraipolis.com/documento/dc_3db7c812ad85

How to cite

When Repository Labels Are Not Image-Level Truth: A Supervision Auditing Framework for Chest Radiograph AI — arXiv:2608.10084 — arXiv (en), https://arxiv.org/abs/2608.10084 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3db7c812ad85 (Metraipolis archive dc_3db7c812ad85, sha256 63c27bf258fba841ca086057801fd7313c562d5ce823dae11fd1018e69485e31)
BibTeX
@misc{dc3db7c812ad85,
  title = {When Repository Labels Are Not Image-Level Truth: A Supervision Auditing Framework for Chest Radiograph AI — arXiv:2608.10084},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.10084},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3db7c812ad85; sha256 63c27bf258fba841ca086057801fd7313c562d5ce823dae11fd1018e69485e31},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.10084. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Public chest X-ray repositories are widely used to train medical AI systems, yet their labels are typically extracted from radiology reports rather than verified directly on images. As a result, repository labels are often treated as image-level ground truth without validating whether they reflect what is actually visible in the radiograph. We introduce Repository Supervision Auditing (RSA), a framework that evaluates repository-derived labels against expert image-level annotations before model development. Using cardiomegaly in MIMIC-CXR as a case study, RSA compares repository labels with radiologist-reviewed image annotations, characterizes disagreement sources, and builds a curated cohort for deployment-oriented evaluation. Repository-derived cardiomegaly labels showed near-zero agreement with expert image-level assessment, identifying only 1% of expert-confirmed cases. Most discrepancies resulted from non-mention rather than explicit report negation, with expert-confirmed cardiomegaly identified in nearly half of studies assigned a repository-derived No Finding label. Using the resulting expert-curated cohort, a DenseNet121 model achieved a test ROC-AUC of 0.853. These findings show that repository labels may not reliably represent image-level truth and highlight supervision auditing as a critical step for developing trustworthy medical imaging AI.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-12 in cs.CV).
Categorie arXiv: Image and Video Processing (eess.IV) ; Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2608.10084 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T06:25:27.344Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-08-12?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).