Metraipolis › archive › dc_4d479ccdd787
Which Objectives Need a Dial? Predicting Objective Conflict and Covering Trade-offs in Steerable Pluralistic Alignment — arXiv:2609.26929
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.26929
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T09:10:59.334Z
- SHA-256 of the archived text
a81d2f7b3a3667ac0bb946a5c708ced76196acd4883956cedee91c8ad7d07a9d- Stable URL of this page
https://metraipolis.com/documento/dc_4d479ccdd787
How to cite
Which Objectives Need a Dial? Predicting Objective Conflict and Covering Trade-offs in Steerable Pluralistic Alignment — arXiv:2609.26929 — arXiv (en), https://arxiv.org/abs/2609.26929 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4d479ccdd787 (Metraipolis archive dc_4d479ccdd787, sha256 a81d2f7b3a3667ac0bb946a5c708ced76196acd4883956cedee91c8ad7d07a9d)
BibTeX
@misc{dc4d479ccdd787,
title = {Which Objectives Need a Dial? Predicting Objective Conflict and Covering Trade-offs in Steerable Pluralistic Alignment — arXiv:2609.26929},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.26929},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4d479ccdd787; sha256 a81d2f7b3a3667ac0bb946a5c708ced76196acd4883956cedee91c8ad7d07a9d},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.26929. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
People hold diverse, sometimes conflicting values, so no single aligned model can satisfy everyone. Pluralistic alignment therefore calls for steerable models that can balance competing objectives differently. Multi-Objective Direct Preference Optimization (MODPO) does this by using an objective weight to span a continuum of trade-offs. We study two questions: when can one model improve two objectives simultaneously, and how can many trade-offs be covered without training a separate model for each? Across seven objective pairs from HelpSteer and UltraFeedback, two pre-training measurements predict whether objectives align or conflict for human-annotated data, but not for AI-annotated data, where response length and repetition confound reward-model scores. For broader trade-off coverage, selecting the nearest trained model and merging model parameters both help, but neither consistently matches direct training. These findings yield practical guidance for building steerable models that serve diverse preferences. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-24 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL); Computers and Society (cs.CY). Pagina del lavoro: https://arxiv.org/abs/2609.26929 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T08:13:26.456Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).