Metraipolis › archive › dc_a2d8adcc144b

Beyond Average Safety: Chance-Constrained LLM Fine-tuning — arXiv:2609.29960

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.29960
License (as archived)
CC0-1.0
Archived
2026-10-08T04:27:54.077Z
SHA-256 of the archived text
dbb8ae1c535802d0717578c731862ddca6581d3eea73161673c717d9bda9a441
Stable URL of this page
https://metraipolis.com/documento/dc_a2d8adcc144b

How to cite

Beyond Average Safety: Chance-Constrained LLM Fine-tuning — arXiv:2609.29960 — arXiv (en), https://arxiv.org/abs/2609.29960 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a2d8adcc144b (Metraipolis archive dc_a2d8adcc144b, sha256 dbb8ae1c535802d0717578c731862ddca6581d3eea73161673c717d9bda9a441)
BibTeX
@misc{dca2d8adcc144b,
  title = {Beyond Average Safety: Chance-Constrained LLM Fine-tuning — arXiv:2609.29960},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.29960},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a2d8adcc144b; sha256 dbb8ae1c535802d0717578c731862ddca6581d3eea73161673c717d9bda9a441},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.29960. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Fine-tuning large language models on new objectives can improve helpfulness, instruction following, or domain-specific performance, but it can also induce regressions on safety-critical prompts. Existing safety-preserving fine-tuning methods typically control average safety loss or use weighted auxiliary penalties, which can obscure rare but severe failures. We propose a chance-constrained formulation for safety-preserving fine-tuning that limits the fraction of safety examples whose degradation relative to a reference model exceeds a prescribed threshold. Because the resulting empirical chance constraint contains a discontinuous indicator, we introduce a differentiable majorization of the violation rate, yielding a tractable conservative constraint. We then develop a constraint-aware gradient descent method that treats the majorized constraint as a safe set in parameter space and minimally modifies the fine-tuning direction to preserve feasibility. The resulting update admits a closed form and produces a tail-aware safety correction that emphasizes examples near or above the degradation threshold. We conduct an extensive set of experiments on harmful fine-tuning across three different tasks and three models and show that our approach consistently outperforms the baselines that exist in the literature. These results suggest that safety preservation in LLM fine-tuning is better viewed as a reliability-constrained optimization problem than as average-risk regularization.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-25 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.29960 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T03:50:58.194Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).