Metraipolis › archive › dc_72d57504b37c

Temperature, Not Knowledge: What Recursive Self-Training and Its Remedies Do to a Language Model's Confidence — arXiv:2611.01059

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.01059
License (as archived): CC0-1.0
Archived: 2026-11-03T11:01:35.936Z
SHA-256 of the archived text: d922eb2b749bfadb62e78c80489c072a6111172b58b3d515652657e6dca25e6e
Stable URL of this page: https://metraipolis.com/documento/dc_72d57504b37c

How to cite

Temperature, Not Knowledge: What Recursive Self-Training and Its Remedies Do to a Language Model's Confidence — arXiv:2611.01059 — arXiv (en), https://arxiv.org/abs/2611.01059 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_72d57504b37c (Metraipolis archive dc_72d57504b37c, sha256 d922eb2b749bfadb62e78c80489c072a6111172b58b3d515652657e6dca25e6e)

BibTeX

@misc{dc72d57504b37c,
  title = {Temperature, Not Knowledge: What Recursive Self-Training and Its Remedies Do to a Language Model's Confidence — arXiv:2611.01059},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.01059},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_72d57504b37c; sha256 d922eb2b749bfadb62e78c80489c072a6111172b58b3d515652657e6dca25e6e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01059. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Recursive self-training can collapse language models, and many proposed remedies target overconfidence, yet their effects on confidence are rarely measured. We study a five-round self-training loop on Qwen3-1.7B, using three seeds per compared method, and distinguish confidence temperature (drift from a calibrated starting point) from discrimination (ranking correct answers above incorrect ones). Accuracy and overall discrimination remain nearly unchanged, but confidence sharpens and the most confident answers become less reliable. Matching predictive entropy to the calibrated starting model each round recalibrates nearly as well as label-based temperature fitting, at both 1.7B and 8B. Sharpening is not inherent to self-training: exact samples from a model's distribution have zero expected gradient. Instead, most drift comes from fitting each generated answer twice, with top-p truncation contributing the rest; removing both keeps entropy on generated outputs flat. Remedies affect the two properties differently: truncated cross-entropy, an entropy bonus, and single-pass training preserve temperature without improving discrimination, while focal loss and multi-sample targets improve discrimination but cause underconfidence. Reference-data mixing also overshoots. Pairing a ranking-improving loss with label-free recalibration achieves both.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.01059 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:59:45.754Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).