Metraipolis › archive › dc_bac40287d1e7

What Paired Evaluations Reveal under Visual Perturbations — arXiv:2609.35583

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.35583
License (as archived)
CC0-1.0
Archived
2026-10-07T23:03:49.829Z
SHA-256 of the archived text
f9a6b97c26de25431c65ed3854837dd4519069ba3fceb5bb71a68807b37eb8b0
Stable URL of this page
https://metraipolis.com/documento/dc_bac40287d1e7

How to cite

What Paired Evaluations Reveal under Visual Perturbations — arXiv:2609.35583 — arXiv (en), https://arxiv.org/abs/2609.35583 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bac40287d1e7 (Metraipolis archive dc_bac40287d1e7, sha256 f9a6b97c26de25431c65ed3854837dd4519069ba3fceb5bb71a68807b37eb8b0)
BibTeX
@misc{dcbac40287d1e7,
  title = {What Paired Evaluations Reveal under Visual Perturbations — arXiv:2609.35583},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.35583},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bac40287d1e7; sha256 f9a6b97c26de25431c65ed3854837dd4519069ba3fceb5bb71a68807b37eb8b0},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.35583. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Robustness evaluation must examine diverse visual perturbations, while benchmarks cover only some real-world conditions and physical testing is costly. Paired evaluations link clean and perturbed predictions for the same image, capturing changes in correctness, confidence, and acceptance beyond aggregate accuracy. We investigate how this image correspondence supports two needs in robustness evaluation: interpreting paired evaluation results and prioritizing samples for physical testing. To interpret paired evaluation results, we fix both sets of prediction records and vary their correspondence within each class. We prove that classwise correct-correct counts give the same sharp bounds on lost acceptance and mean true-class probability decrease among retained-correct inputs as any feasible five-state refinement. Distinguishing persistent from changed wrong answers can further constrain accepted-error transitions, while shared correspondence can establish policy orderings left unresolved by separate cost intervals. To prioritize samples for physical testing, we retain each image's synthetic responses and rank clean-correct images by their mean true-class probability under corruption. Across 44 classifiers, testing the highest-risk 20% finds 67% and 45% of failures under mild screen and print recaptures, versus 58% and 36% for clean confidence and 60% and 37% for an equal-size natural-transformation average. With both probability averaging and an A3Rank scoring adaptation, the tested corruption set yields higher mean failure recall than the natural-transform set; differences between scores depend on the source and budget. Together, these findings show that the value of correspondence depends on the evaluation objective: classwise counts suffice for specified reliability bounds, while image-specific synthetic responses improve the allocation of physical tests within the evaluated pool.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-29 in stat.AP).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Applications (stat.AP).
Pagina del lavoro: https://arxiv.org/abs/2609.35583 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T22:42:04.186Z dalla pagina https://arxiv.org/catchup/stat.AP/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).