Metraipolis › archive › dc_bdd30e9fffe5
Doomed to Re-Annotate, Forever: The ImageNet Story — arXiv:2608.13783
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.13783
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T04:02:10.989Z
- SHA-256 of the archived text
224c866f8c630fb2bad043a17e9ffb06b031924e576c415e7602257f8d3a02af- Stable URL of this page
https://metraipolis.com/documento/dc_bdd30e9fffe5
How to cite
Doomed to Re-Annotate, Forever: The ImageNet Story — arXiv:2608.13783 — arXiv (en), https://arxiv.org/abs/2608.13783 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bdd30e9fffe5 (Metraipolis archive dc_bdd30e9fffe5, sha256 224c866f8c630fb2bad043a17e9ffb06b031924e576c415e7602257f8d3a02af)
BibTeX
@misc{dcbdd30e9fffe5,
title = {Doomed to Re-Annotate, Forever: The ImageNet Story — arXiv:2608.13783},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.13783},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bdd30e9fffe5; sha256 224c866f8c630fb2bad043a17e9ffb06b031924e576c415e7602257f8d3a02af},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.13783. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Top-1 accuracy on ImageNet-1k remains the most commonly reported metric in visual recognition. Quality issues with the dataset have been repeatedly reported, yet the original 2012 noisy labels are still predominantly used. The paper presents a comprehensive effort, which goes well beyond prior correction attempts, towards obtaining accurate and complete ImageNet-1k validation set annotations. The result, ReImageNet, includes multilabel correction, object localization, revised class definitions, and semantic attributes (text-recognition, rendition, reflection, crowd, dominant). The reannotation reveals that approximately 12% of the original ImageNet-1k labels are incorrect, 33.3% of images are multilabel and 3.8% contain no object from an ImageNet-1k class. With the new labels, top-1 accuracy increases by up to 1.2% for supervised models and by 5-6% for MLLMs. We argue that annotation at ImageNet scale cannot realistically be completed in one pass, as errors and definitional issues are discovered only through annotating, and we build our pipeline around repeated refinement and error checking. We observed that human and LLM collaboration with appropriate tooling represents the current quality ceiling for annotation at this scale. ImageNet-1k issues propagate into its derivative test sets, indicating that the problem is structural rather than specific to any single benchmark. All annotations, class definitions, guidelines, and analysis code have been publicly released. Project page: this https URL Annotations: this https URL Code: this https URL Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-17 in cs.CV). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2608.13783 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T03:57:57.758Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-08-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).