Metraipolis › archive › dc_ab47fede7352

Scores That Hold, Benchmarks That Leak: Measuring Dataset Contamination in Public Brain-Tumor MRI Classification — arXiv:2610.00421

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.00421
License (as archived)
CC0-1.0
Archived
2026-10-06T12:09:21.407Z
SHA-256 of the archived text
9caeea7d748fb8821b014ba3848606c8c9ebe2fda50eba6b52cc3f0955369adf
Stable URL of this page
https://metraipolis.com/documento/dc_ab47fede7352

How to cite

Scores That Hold, Benchmarks That Leak: Measuring Dataset Contamination in Public Brain-Tumor MRI Classification — arXiv:2610.00421 — arXiv (en), https://arxiv.org/abs/2610.00421 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ab47fede7352 (Metraipolis archive dc_ab47fede7352, sha256 9caeea7d748fb8821b014ba3848606c8c9ebe2fda50eba6b52cc3f0955369adf)
BibTeX
@misc{dcab47fede7352,
  title = {Scores That Hold, Benchmarks That Leak: Measuring Dataset Contamination in Public Brain-Tumor MRI Classification — arXiv:2610.00421},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.00421},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ab47fede7352; sha256 9caeea7d748fb8821b014ba3848606c8c9ebe2fda50eba6b52cc3f0955369adf},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.00421. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Automated classification of brain tumors from MRI is a heavily published application of deep learning in medical imaging, with reported accuracies on public benchmarks routinely exceeding 98%. However, accuracy does not capture a critical dimension of benchmark quality: dataset integrity, defined as the independence of test from training data at the image, patient, and acquisition-source levels. We introduce a three-layer contamination framework comprising duplicate, patient, and source-label leakage to assess the public corpora on which this literature rests. We audit the three most widely used corpora against a chest-radiograph negative control and quantify each layer's effect on measured performance across nine architectures and three evaluation conditions. Contamination is severe at every layer: 28.8% of the dominant corpus's official test split has a near-twin in its own training split, a second corpus leaks 22.3% of its test images byte-identically, 95.5% of traceable test images share a patient with training, and file-header features containing no anatomy separate tumor from no-tumor at 0.959 balanced accuracy, at parity with fine-tuned ResNet backbones. The unexpected result is that removing every identified leaked test image leaves balanced accuracy essentially unchanged: stable performance after deduplication does not establish benchmark integrity. Our findings establish dataset integrity as a distinct, measurable axis of benchmark quality that a stable leaderboard cannot certify. For biomedical research, reported accuracy on these corpora alone does not establish that a model has learned to recognize tumors rather than exploit dataset-specific cues. We release the contaminated-file lists, recovered patient identifiers, and deduplicated splits.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-02 in cs.AI).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.00421 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T10:21:28.064Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).