Metraipolis › archive › dc_cc52dccbf17a

Temperature, Not Knowledge: What Recursive Self-Training and Its Remedies Do to a Language Model's Confidence — arXiv:2611.01059

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.01059
License (as archived): CC0-1.0
Archived: 2026-11-03T10:31:40.539Z
SHA-256 of the archived text: 554a0efec26d4518c723d767d82c41d251385b097a5f05b80e24dd882400310f
Stable URL of this page: https://metraipolis.com/documento/dc_cc52dccbf17a

How to cite

Temperature, Not Knowledge: What Recursive Self-Training and Its Remedies Do to a Language Model's Confidence — arXiv:2611.01059 — arXiv (en), https://arxiv.org/abs/2611.01059 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cc52dccbf17a (Metraipolis archive dc_cc52dccbf17a, sha256 554a0efec26d4518c723d767d82c41d251385b097a5f05b80e24dd882400310f)

BibTeX

@misc{dccc52dccbf17a,
  title = {Temperature, Not Knowledge: What Recursive Self-Training and Its Remedies Do to a Language Model's Confidence — arXiv:2611.01059},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.01059},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cc52dccbf17a; sha256 554a0efec26d4518c723d767d82c41d251385b097a5f05b80e24dd882400310f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01059. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Recursive self-training can collapse language models, and many proposed remedies target overconfidence, yet their effects on confidence are rarely measured. We study a five-round self-training loop on Qwen3-1.7B, using three seeds per compared method, and distinguish confidence temperature (drift from a calibrated starting point) from discrimination (ranking correct answers above incorrect ones). Accuracy and overall discrimination remain nearly unchanged, but confidence sharpens and the most confident answers become less reliable. Matching predictive entropy to the calibrated starting model each round recalibrates nearly as well as label-based temperature fitting, at both 1.7B and 8B. Sharpening is not inherent to self-training: exact samples from a model's distribution have zero expected gradient. Instead, most drift comes from fitting each generated answer twice, with top-p truncation contributing the rest; removing both keeps entropy on generated outputs flat. Remedies affect the two properties differently: truncated cross-entropy, an entropy bonus, and single-pass training preserve temperature without improving discrimination, while focal loss and multi-sample targets improve discrimination but cause underconfidence. Reference-data mixing also overshoots. Pairing a ranking-improving loss with label-free recalibration achieves both.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.01059 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:29:48.491Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).