Metraipolis › archive › dc_5d91c9c0715a
How Many Labels Does Model Choice Need? Certificates and Budgets for Selective Prediction — arXiv:2609.18622
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.18622
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T15:04:50.610Z
- SHA-256 of the archived text
57b69b9e1e94c6728f9cc961a19f02076aa9b7a8e0f472a76ad7798a466dd1c7- Stable URL of this page
https://metraipolis.com/documento/dc_5d91c9c0715a
How to cite
How Many Labels Does Model Choice Need? Certificates and Budgets for Selective Prediction — arXiv:2609.18622 — arXiv (en), https://arxiv.org/abs/2609.18622 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5d91c9c0715a (Metraipolis archive dc_5d91c9c0715a, sha256 57b69b9e1e94c6728f9cc961a19f02076aa9b7a8e0f472a76ad7798a466dd1c7)
BibTeX
@misc{dc5d91c9c0715a,
title = {How Many Labels Does Model Choice Need? Certificates and Budgets for Selective Prediction — arXiv:2609.18622},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.18622},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5d91c9c0715a; sha256 57b69b9e1e94c6728f9cc961a19f02076aa9b7a8e0f472a76ad7798a466dd1c7},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.18622. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Classifiers can make identical predictions yet require labels to compare their selective performance: confidence ranks weight the same errors differently. We quantify this requirement for the area under the generalized risk-coverage curve (AUGRC). A prelabel lower bound rules out insufficient budgets. With all labels known, a covering linear program bounds the minimum number of labels sufficient to fix the winner (the certificate size) within $K-1$ labels for $K$ candidates. For fixed $K$, independent uniform orders and identical predictions, the prelabel bound approaches one quarter of the pool. With iid Bernoulli errors independent of the orders, every exact acquisition policy reads almost all labels asymptotically, although a two-candidate certificate needs only half. Across 108 feature-panel comparisons on nine datasets, disagreement labels settle every accuracy choice but no AUGRC choice. A 20% budget is ruled out in 96 conditions; certificates need 56-57% on average. On ten conditions with pretrained image classifiers, confidence-score choice reads 68-91% of 10,000 labels for exact selection and 50-67% with AUGRC tolerance $5\times10^{-4}$. An exact stopping test works with any acquisition order. Together, these results link confidence ranks to label budgets and certified model comparison.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-17 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2609.18622 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T14:08:19.141Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).