Metraipolis › archive › dc_c561b32aef97

Geometry of Values: Task Vector Composition for Ethical Preference Alignment in Language Models — arXiv:2609.21094

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.21094
License (as archived)
CC0-1.0
Archived
2026-10-10T01:07:49.749Z
SHA-256 of the archived text
54f7559bfcc3a52fc1323075aa36276126b08ac7b2079a244c345f502d7f04e9
Stable URL of this page
https://metraipolis.com/documento/dc_c561b32aef97

How to cite

Geometry of Values: Task Vector Composition for Ethical Preference Alignment in Language Models — arXiv:2609.21094 — arXiv (en), https://arxiv.org/abs/2609.21094 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c561b32aef97 (Metraipolis archive dc_c561b32aef97, sha256 54f7559bfcc3a52fc1323075aa36276126b08ac7b2079a244c345f502d7f04e9)
BibTeX
@misc{dcc561b32aef97,
  title = {Geometry of Values: Task Vector Composition for Ethical Preference Alignment in Language Models — arXiv:2609.21094},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.21094},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c561b32aef97; sha256 54f7559bfcc3a52fc1323075aa36276126b08ac7b2079a244c345f502d7f04e9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.21094. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large Language Models (LLMs) are increasingly deployed in applications that must weigh clashing moral values, yet even strong models exhibit hidden biases and brittle instruction-following across languages. We introduce a 12,000-instance dataset of two-option dilemmas covering pairwise three value conflicts: Honesty vs. Justice, Justice vs. Autonomy, and Autonomy vs. Honesty, along with their translations into Hindi, Arabic, Spanish, and Chinese, to probe cross-lingual behavior. Benchmarking on GPT-5-mini reveals that it consistently favors Honesty over Autonomy across all five languages when no policy is given. The Llama-3.2-1/3B models exhibit strong first-option bias; however, both plain fine-tuning and Direct Preference Optimization fine-tuning effectively remove this bias, increasing accuracy to greater than 98%. In order to decouple the effect of learning correlations in the dataset from abstract values, we propose a task vector transfer based experiment where after computing the task vectors for a direction of value preference we orthogonalize it with respect to the general instruction following vector. Our experiment shows that this method is effective in isolating the direction of the specific value preference that can successfully be used to conduct task arithmetic to obtain a model with the opposite stance.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-21 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.21094 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-10T00:23:42.507Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-21?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).