Metraipolis › archive › dc_c970462958d7

Learning Style, Forgetting Semantics: A Case Study of SFT and RFT on Classification Tasks — arXiv:2610.02437

Source
arXiv
Original
https://arxiv.org/abs/2610.02437
License (as archived)
CC0-1.0
Archived
2026-10-05T22:50:00.668Z
SHA-256 of the archived text
59acf3424e990c076c7debafd5b8b1997d407fefc54c5398ebe20c8000f849ed
Stable URL of this page
https://metraipolis.com/documento/dc_c970462958d7

How to cite

Learning Style, Forgetting Semantics: A Case Study of SFT and RFT on Classification Tasks — arXiv:2610.02437 — arXiv, https://arxiv.org/abs/2610.02437 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c970462958d7 (Metraipolis archive dc_c970462958d7, sha256 59acf3424e990c076c7debafd5b8b1997d407fefc54c5398ebe20c8000f849ed)
BibTeX
@misc{dcc970462958d7,
  title = {Learning Style, Forgetting Semantics: A Case Study of SFT and RFT on Classification Tasks — arXiv:2610.02437},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.02437},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c970462958d7; sha256 59acf3424e990c076c7debafd5b8b1997d407fefc54c5398ebe20c8000f849ed},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.02437. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Why does supervised fine-tuning (SFT) lead to more forgetting than reinforcement fine-tuning (RFT), even when all teacher demonstrations are semantically correct? We study this question on classification tasks where tokens within each semantic class express the same semantic answer in different styles. The tasks share an underlying semantic rule but differ in their prompt distributions and teachers' stylistic preferences. Using a tractable linear-softmax policy, we derive an exact decomposition of the updates into semantic and style components. We show that, at a common policy and prompt, SFT and RFT have parallel semantic updates but differ in their style dynamics. Starting from a policy with no within-class style preference, RFT with exact policy gradients preserves this symmetry, whereas SFT with a nonuniform teacher develops off-axis style drift along a nonzero task mean under population updates. We use this drift to establish a separation under explicit conditions: for population updates from a common perfectly fitted checkpoint, SFT forgetting admits a strictly positive lower bound over a finite training interval, while RFT retains zero semantic error. Simulations over task sequences support these theoretical predictions.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (stat.ML) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.02437 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-05T22:45:55.754Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).