Metraipolis › archive › dc_bc7df3436d83

Minimal Superior Penalties: Characterising and Calibrating Decision-Aware Proper Scoring Rules — arXiv:2611.03734

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.03734
License (as archived)
CC0-1.0
Archived
2026-11-04T21:50:45.690Z
SHA-256 of the archived text
7f6dbf5e8e33bafa2aa5ab7e4540b9f55e3d8de215068f6dcb3d7b3e63669782
Stable URL of this page
https://metraipolis.com/documento/dc_bc7df3436d83

How to cite

Minimal Superior Penalties: Characterising and Calibrating Decision-Aware Proper Scoring Rules — arXiv:2611.03734 — arXiv (en), https://arxiv.org/abs/2611.03734 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bc7df3436d83 (Metraipolis archive dc_bc7df3436d83, sha256 7f6dbf5e8e33bafa2aa5ab7e4540b9f55e3d8de215068f6dcb3d7b3e63669782)
BibTeX
@misc{dcbc7df3436d83,
  title = {Minimal Superior Penalties: Characterising and Calibrating Decision-Aware Proper Scoring Rules — arXiv:2611.03734},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.03734},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bc7df3436d83; sha256 7f6dbf5e8e33bafa2aa5ab7e4540b9f55e3d8de215068f6dcb3d7b3e63669782},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.03734. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Training a classifier means letting a validation metric decide which model is kept and when training stops. A natural requirement on that metric is superiority: every correctly classified forecast should score better than every incorrectly classified one, however close the underlying probabilities are. Neither the Brier score nor the logarithmic score satisfies this on its own. We show that superiority is governed by the separation between two endpoints -- the supremum of the score on correct predictions and the infimum on incorrect predictions -- and derive both in closed form for each score. Their difference is the infimal penalty, the greatest lower bound on the penalty needed to restore superiority. Prior work restores superiority with a penalty calibrated from the first endpoint alone; for the Brier base, our infimal penalty is smaller than that published penalty by a factor ranging from 4.00 at three classes to 2.11 at twenty. We further embed the published and infimal penalties in a one-parameter family and show that, under a single-valued tie convention, every member of the family is strictly proper. In a pre-specified two-dataset case study, mean selected-model macro-F1 rises with the penalty in the reported ten-class regimes, though not distinguishably from noise once seed/selection-mode nesting is accounted for; a fifth regime at forty-seven classes does not reproduce even this descriptive pattern. The intended no-superiority baseline is algebraically a rescaling of the family under test, so the experiment cannot attribute the observed pattern specifically to superiority. When superiority itself is the objective, any penalty strictly above the infimal value suffices; choosing a larger penalty is a separate calibration trade-off rather than a requirement for the guarantee.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2611.03734 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T21:38:49.665Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).