Metraipolis › archive › dc_f019f2b55868

Which Tokens Should SFT Actually Learn? A Token-Trimming Perspective on Mathematical Reasoning — arXiv:2609.09707

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.09707
License (as archived)
CC0-1.0
Archived
2026-10-16T07:24:30.439Z
SHA-256 of the archived text
e16b4ee95ba7b7e5542a6d44989a573331f82bf3424d7dd25ab7fc2e16976b93
Stable URL of this page
https://metraipolis.com/documento/dc_f019f2b55868

How to cite

Which Tokens Should SFT Actually Learn? A Token-Trimming Perspective on Mathematical Reasoning — arXiv:2609.09707 — arXiv (en), https://arxiv.org/abs/2609.09707 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f019f2b55868 (Metraipolis archive dc_f019f2b55868, sha256 e16b4ee95ba7b7e5542a6d44989a573331f82bf3424d7dd25ab7fc2e16976b93)
BibTeX
@misc{dcf019f2b55868,
  title = {Which Tokens Should SFT Actually Learn? A Token-Trimming Perspective on Mathematical Reasoning — arXiv:2609.09707},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.09707},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f019f2b55868; sha256 e16b4ee95ba7b7e5542a6d44989a573331f82bf3424d7dd25ab7fc2e16976b93},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.09707. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Supervised fine-tuning (SFT) applies a uniform cross-entropy loss to all target tokens, even though different tokens provide unequal learning signals for mathematical reasoning. This uniform treatment can over-sharpen already mastered tokens while amplifying learning pressure on uncertain, low-confidence tokens, leading to suboptimal training dynamics. We propose Trimmed Logit-Gap SFT (TrimSFT), a simple token-level reweighting method that scales the SFT loss according to the logit gap between the gold token and its strongest competitor. TrimSFT trims supervision away from both extremes: tokens already mastered (large logit gap) and tokens weakly supported by the current model (small or negative logit gap), concentrating learning within an intermediate logit-gap region between them. We instantiate this principle with a Gaussian weight centered at margin m with bandwidth {\tau}, requiring no reference model or additional forward pass. We evaluate TrimSFT on six base models from the Llama, Qwen, and DeepMath families across five mathematical reasoning benchmarks. TrimSFT consistently improves over standard SFT, achieving the best average performance on five out of six models, with gains of up to +26.9 points over SFT on MATH500. Further analyses show that the bandwidth {\tau} matters more than the exact margin location, and that half-trim variants that remove supervision pressure from only one side yield inferior trade-offs. A token-level logit-gap distribution analysis suggests that TrimSFT reshapes model confidence in a more balanced way than uniform SFT or monotonic reweighting methods. These results suggest that reasoning SFT can benefit from trimming both extremes rather than treating all tokens uniformly.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-10 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.09707 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T07:00:16.210Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-10?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).