Metraipolis › archive › dc_934edff4b561

How Much Can Reliability Drift Under a Fixed Confidence Distribution? — arXiv:2609.38917

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.38917
License (as archived)
CC0-1.0
Archived
2026-10-07T12:42:20.371Z
SHA-256 of the archived text
dd82388870470f6b1457f5af0d39afd4e291f77d423a7fc50325d3411dd171f3
Stable URL of this page
https://metraipolis.com/documento/dc_934edff4b561

How to cite

How Much Can Reliability Drift Under a Fixed Confidence Distribution? — arXiv:2609.38917 — arXiv (en), https://arxiv.org/abs/2609.38917 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_934edff4b561 (Metraipolis archive dc_934edff4b561, sha256 dd82388870470f6b1457f5af0d39afd4e291f77d423a7fc50325d3411dd171f3)
BibTeX
@misc{dc934edff4b561,
  title = {How Much Can Reliability Drift Under a Fixed Confidence Distribution? — arXiv:2609.38917},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.38917},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_934edff4b561; sha256 dd82388870470f6b1457f5af0d39afd4e291f77d423a7fc50325d3411dd171f3},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.38917. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

A classifier's conditional accuracy can change while its confidence distribution stays exactly the same. We study the worst-case movement of the reliability relation under covariate shifts that preserve the distribution of the confidence score, constraining the reweighting within each confidence level by a $\chi^2$ budget; the resulting worst case, as a function of the budget, is a fragility profile. On an interval of budgets that can be computed from the source distribution, the profile equals exactly the square root of the budget times the within-level variance of the correctness propensity -- the grouping-loss term of calibration-refinement decompositions. Beyond this interval the profile is governed by the tails of the propensity law, and the entire upward profile determines the centred within-level law; consequently, calibration residual and grouping variance do not determine fragility in general, though they do when labels and predictions are deterministic. Since the propensity is not observed, we restrict reweightings to a learned finite readout within confidence bins, bound the part the restriction misses by the grouping variance remaining inside readout cells, estimate the restricted profile with role-separated labels, and provide a separate split-sample lower confidence bound. On ImageNet this bound is positive in both splits for four of six primary classifiers and nine of twelve additional ones as released, and for three of eighteen after temperature scaling. Held-out drift under optimised reweightings fitted without evaluation labels tracks the estimated profile; an exploratory label-permutation diagnostic yields near-zero agreement for this statistic while largely reproducing the correlation observed for unsigned random reweightings.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.38917 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T12:14:13.711Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).