Metraipolis › archive › dc_503bf40f7112

Decoding-time Realignment of Language Models — arXiv:2402.02992

Source
arXiv (en)
Original
https://arxiv.org/abs/2402.02992
License (as archived)
CC0-1.0
Archived
2026-10-06T09:32:32.845Z
SHA-256 of the archived text
1400aa9456babff76b9ca93ffed242d69afac532dadf0f8675df10286d1ac84b
Stable URL of this page
https://metraipolis.com/documento/dc_503bf40f7112

How to cite

Decoding-time Realignment of Language Models — arXiv:2402.02992 — arXiv (en), https://arxiv.org/abs/2402.02992 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_503bf40f7112 (Metraipolis archive dc_503bf40f7112, sha256 1400aa9456babff76b9ca93ffed242d69afac532dadf0f8675df10286d1ac84b)
BibTeX
@misc{dc503bf40f7112,
  title = {Decoding-time Realignment of Language Models — arXiv:2402.02992},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2402.02992},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_503bf40f7112; sha256 1400aa9456babff76b9ca93ffed242d69afac532dadf0f8675df10286d1ac84b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2402.02992. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Aligning language models with human preferences is crucial for reducing errors and biases in these models. Alignment techniques, such as reinforcement learning from human feedback (RLHF), are typically cast as optimizing a tradeoff between human preference rewards and a proximity regularization term that encourages staying close to the unaligned model. Selecting an appropriate level of regularization is critical: insufficient regularization can lead to reduced model capabilities due to reward hacking, whereas excessive regularization hinders alignment. Traditional methods for finding the optimal regularization level require retraining multiple models with varying regularization strengths. This process, however, is resource-intensive, especially for large models. To address this challenge, we propose decoding-time realignment (DeRa), a simple method to explore and evaluate different regularization strengths in aligned models without retraining. DeRa enables control over the degree of alignment, allowing users to smoothly transition between unaligned and aligned models. It also enhances the efficiency of hyperparameter tuning by enabling the identification of effective regularization strengths using a validation dataset.

Tema degli esploratori: statistica e scienza dei dati (elenco arXiv stat.ML, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2402.02992 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T08:24:13.953Z dalla pagina https://arxiv.org/list/stat.ML/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).