Metraipolis › archive › dc_8955fc11dddc

Beyond Distribution Matching: Selective Margin Preservation for Supervised Fine-Tuning — arXiv:2610.19685

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.19685
License (as archived)
CC0-1.0
Archived
2026-10-15T14:17:43.551Z
SHA-256 of the archived text
bb70b4b95d6f6bed990d3a7317d1efba9e5c54cb1e532801ef53abe44d7c96d4
Stable URL of this page
https://metraipolis.com/documento/dc_8955fc11dddc

How to cite

Beyond Distribution Matching: Selective Margin Preservation for Supervised Fine-Tuning — arXiv:2610.19685 — arXiv (en), https://arxiv.org/abs/2610.19685 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8955fc11dddc (Metraipolis archive dc_8955fc11dddc, sha256 bb70b4b95d6f6bed990d3a7317d1efba9e5c54cb1e532801ef53abe44d7c96d4)
BibTeX
@misc{dc8955fc11dddc,
  title = {Beyond Distribution Matching: Selective Margin Preservation for Supervised Fine-Tuning — arXiv:2610.19685},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.19685},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8955fc11dddc; sha256 bb70b4b95d6f6bed990d3a7317d1efba9e5c54cb1e532801ef53abe44d7c96d4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.19685. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Supervised fine-tuning (SFT) changes logit margins among non-target tokens at teacher-forced prefixes. Matching a local reference distribution keeps those probabilities and also penalizes margin expansions. We ask whether a soft auxiliary can preserve selected local non-target margin relations without freezing that distribution. We introduce \textbf{Selective Rank-Preserving SFT (SRP-SFT)}, a soft margin target: a hinge on reference-scaled lower bounds for pairs from the reference's top-$K$ non-target tokens, applied at fixed teacher-forced prefixes. The hinge stops once the requested bound is met. We derive the KL cost of violating a margin, characterize the feasible region and its conditional utility, and give a finite-step condition for reducing aggregate violations. In one-epoch experiments on Qwen3-4B-Base and UltraFeedback, this target raises MATH-500 and code-generation scores over CE and LP-SFT at all three training seeds. At seed 1234, MATH-500 pass@1 rises from 49.56\% to 54.56\% and the four-metric code mean rises from 65.91\% to 69.50\% relative to LP-SFT. Ranking controls, held-out cache diagnostics, sensitivity sweeps, and six additional backbones show where these score changes appear.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.19685 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T13:53:35.628Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).