Metraipolis › archive › dc_efdf9bbdfbdf

Suan: Rectifying Direct Preference Safety Alignment in Large Language Models — arXiv:2609.08634

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.08634
License (as archived): CC0-1.0
Archived: 2026-10-16T13:58:20.899Z
SHA-256 of the archived text: 8234cec55c7a358b730c090041239c94e4dd752a69a0c5d40f8023e7f597ced4
Stable URL of this page: https://metraipolis.com/documento/dc_efdf9bbdfbdf

How to cite

Suan: Rectifying Direct Preference Safety Alignment in Large Language Models — arXiv:2609.08634 — arXiv (en), https://arxiv.org/abs/2609.08634 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_efdf9bbdfbdf (Metraipolis archive dc_efdf9bbdfbdf, sha256 8234cec55c7a358b730c090041239c94e4dd752a69a0c5d40f8023e7f597ced4)

BibTeX

@misc{dcefdf9bbdfbdf,
  title = {Suan: Rectifying Direct Preference Safety Alignment in Large Language Models — arXiv:2609.08634},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.08634},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_efdf9bbdfbdf; sha256 8234cec55c7a358b730c090041239c94e4dd752a69a0c5d40f8023e7f597ced4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.08634. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Integrating robust safety guardrails into Large Language Models (LLMs) is essential for delivering helpful yet harmless responses. While proprietary systems exhibit reliable safety controls, their underlying methodologies and trade-offs remain largely undisclosed. Achieving comparable security in open-weight models remains a persistent challenge, as post-trained variants frequently suffer from over-refusal and degraded general quality. To overcome these drawbacks, we introduce Suan, a novel preference optimization algorithm. Unlike existing methods, we formulate the optimization objective directly at the gradient level, bypassing the standard variational derivation. As a result, we obtain more interpretable and robust training dynamics. Extensive evaluations across a diverse suite of competitive baselines and benchmarks demonstrate that Suan achieves superior safety alignment while fully preserving response utility.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-09 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.08634 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T13:39:43.055Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).