Metraipolis › archive › dc_362e180e1e46
From Checkpoint Variation to Selection Gains in Supervised Fine-Tuning — arXiv:2609.36569
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.36569
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T16:26:06.298Z
- SHA-256 of the archived text
abf7eb5131b22e207d151aeb49fb3420f1fba55f63fc0986727d1f9952133be4- Stable URL of this page
https://metraipolis.com/documento/dc_362e180e1e46
How to cite
From Checkpoint Variation to Selection Gains in Supervised Fine-Tuning — arXiv:2609.36569 — arXiv (en), https://arxiv.org/abs/2609.36569 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_362e180e1e46 (Metraipolis archive dc_362e180e1e46, sha256 abf7eb5131b22e207d151aeb49fb3420f1fba55f63fc0986727d1f9952133be4)
BibTeX
@misc{dc362e180e1e46,
title = {From Checkpoint Variation to Selection Gains in Supervised Fine-Tuning — arXiv:2609.36569},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.36569},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_362e180e1e46; sha256 abf7eb5131b22e207d151aeb49fb3420f1fba55f63fc0986727d1f9952133be4},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.36569. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Checkpoint selection is a routine decision in supervised fine-tuning (SFT): training produces multiple checkpoints, but only one is retained. Yet fixed-budget comparisons do not by themselves distinguish three empirical claims: whether more validation data improve checkpoint selection, whether a selection rule outperforms validation-loss selection, and whether it improves over simply retaining the final checkpoint. We therefore treat checkpoint selection as a finite-information decision problem. Holding completed training trajectories, candidate checkpoints, and independent test items fixed, we vary the validation budget and separately measure improvement from additional validation data, gain over negative log-likelihood (NLL) selection, and gain over the final checkpoint. Across 60 mathematical SFT trajectories and 19 configurations, increasing the validation budget from 32 to 305-313 examples raises independent-test accuracy by 0.32 percentage points (pp) for generated-accuracy selection and 0.29 pp for checkpoint agreement, with 95% configuration-bootstrap CIs of [0.10, 0.56] and [0.11, 0.50], respectively. At the full validation budget, the two generation-based rules outperform matched NLL selection by 0.71 and 0.85 pp, respectively, while their gains over the final checkpoint remain unresolved. A cross-domain replication on 12 newly trained Commonsense trajectories shows the same qualitative separation: increasing the validation budget from 32 to 1,024 questions improves generated-accuracy and checkpoint-agreement selection by 0.87 and 0.27 pp, while gains over the final checkpoint again remain unresolved. Together, these results show that benefiting from more validation data, outperforming NLL selection, and outperforming the final checkpoint are distinct empirical claims that require separate evidence. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.AI). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.36569 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T15:54:53.863Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).