Metraipolis › archive › dc_9da0c3bd53bf
SupportCal: Label-Free Calibration of Post-Trained LLMs via Reference Support and Corroboration — arXiv:2609.24303
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.24303
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T06:50:40.518Z
- SHA-256 of the archived text
9a6dc815f64213370a94e279b27a4e1b63c2d2f0a6d4c84c55ddf43300ceb478- Stable URL of this page
https://metraipolis.com/documento/dc_9da0c3bd53bf
How to cite
SupportCal: Label-Free Calibration of Post-Trained LLMs via Reference Support and Corroboration — arXiv:2609.24303 — arXiv (en), https://arxiv.org/abs/2609.24303 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_9da0c3bd53bf (Metraipolis archive dc_9da0c3bd53bf, sha256 9a6dc815f64213370a94e279b27a4e1b63c2d2f0a6d4c84c55ddf43300ceb478)
BibTeX
@misc{dc9da0c3bd53bf,
title = {SupportCal: Label-Free Calibration of Post-Trained LLMs via Reference Support and Corroboration — arXiv:2609.24303},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.24303},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_9da0c3bd53bf; sha256 9a6dc815f64213370a94e279b27a4e1b63c2d2f0a6d4c84c55ddf43300ceb478},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.24303. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Post-training often improves task performance but can degrade confidence calibration, leaving post-trained language models (PoLMs) more overconfident than their corresponding pretrained language models (PLMs). Because task-specific labeled calibration data can be costly or unavailable, the corresponding PLM provides a natural label-free reference for post-hoc calibration. Prior agreement-gated PLM-referenced calibration fits a scalar temperature using only examples on which the PoLM and its PLM reference agree, excluding disagreement examples because direct alignment can drive the fitted temperature excessively high and induce under-confidence. We revisit this binary treatment. A controlled reintroduction diagnostic reveals a non monotonic aggregate effect: admitting a moderate fraction of disagreement examples can improve calibration, whereas the benefit diminishes as unit weight inclusion approaches the full disagreement set. We introduce SupportCal, a label-free post-hoc method that retains agreement examples at unit weight and assigns disagreement examples continuous weights based on the own-base PLM's relative support and corroboration from pretrained references selected from a size-compatible candidate pool. We further characterize when the resulting weighted objective admits a finite optimal temperature. Across MedMCQA and MathQA, SupportCal yields lower mean ECE than the agreement-only baseline for nearly all evaluated target-model configurations; supplementary TweetEval Sentiment results show the same pattern on a fixed-label classification task. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-22 in cs.CL). Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.24303 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T06:48:45.140Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).