Metraipolis › archive › dc_ab47fede7352
Scores That Hold, Benchmarks That Leak: Measuring Dataset Contamination in Public Brain-Tumor MRI Classification — arXiv:2610.00421
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.00421
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T12:09:21.407Z
- SHA-256 of the archived text
9caeea7d748fb8821b014ba3848606c8c9ebe2fda50eba6b52cc3f0955369adf- Stable URL of this page
https://metraipolis.com/documento/dc_ab47fede7352
How to cite
Scores That Hold, Benchmarks That Leak: Measuring Dataset Contamination in Public Brain-Tumor MRI Classification — arXiv:2610.00421 — arXiv (en), https://arxiv.org/abs/2610.00421 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ab47fede7352 (Metraipolis archive dc_ab47fede7352, sha256 9caeea7d748fb8821b014ba3848606c8c9ebe2fda50eba6b52cc3f0955369adf)
BibTeX
@misc{dcab47fede7352,
title = {Scores That Hold, Benchmarks That Leak: Measuring Dataset Contamination in Public Brain-Tumor MRI Classification — arXiv:2610.00421},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.00421},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ab47fede7352; sha256 9caeea7d748fb8821b014ba3848606c8c9ebe2fda50eba6b52cc3f0955369adf},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.00421. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Automated classification of brain tumors from MRI is a heavily published application of deep learning in medical imaging, with reported accuracies on public benchmarks routinely exceeding 98%. However, accuracy does not capture a critical dimension of benchmark quality: dataset integrity, defined as the independence of test from training data at the image, patient, and acquisition-source levels. We introduce a three-layer contamination framework comprising duplicate, patient, and source-label leakage to assess the public corpora on which this literature rests. We audit the three most widely used corpora against a chest-radiograph negative control and quantify each layer's effect on measured performance across nine architectures and three evaluation conditions. Contamination is severe at every layer: 28.8% of the dominant corpus's official test split has a near-twin in its own training split, a second corpus leaks 22.3% of its test images byte-identically, 95.5% of traceable test images share a patient with training, and file-header features containing no anatomy separate tumor from no-tumor at 0.959 balanced accuracy, at parity with fine-tuned ResNet backbones. The unexpected result is that removing every identified leaked test image leaves balanced accuracy essentially unchanged: stable performance after deduplication does not establish benchmark integrity. Our findings establish dataset integrity as a distinct, measurable axis of benchmark quality that a stable leaderboard cannot certify. For biomedical research, reported accuracy on these corpora alone does not establish that a model has learned to recognize tumors rather than exploit dataset-specific cues. We release the contaminated-file lists, recovered patient identifiers, and deduplicated splits. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-02 in cs.AI). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.00421 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-06T10:21:28.064Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).