Metraipolis › archive › dc_7ce1ffa67750

Towards Multi-Value Alignment via Value Localization in LLMs — arXiv:2611.02503

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.02503
License (as archived): CC0-1.0
Archived: 2026-11-03T11:02:42.207Z
SHA-256 of the archived text: 84b05dd148ad8097f66e936799ddb2d89f8beda66dcecb6b0e0fec30a0b5720a
Stable URL of this page: https://metraipolis.com/documento/dc_7ce1ffa67750

How to cite

Towards Multi-Value Alignment via Value Localization in LLMs — arXiv:2611.02503 — arXiv (en), https://arxiv.org/abs/2611.02503 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7ce1ffa67750 (Metraipolis archive dc_7ce1ffa67750, sha256 84b05dd148ad8097f66e936799ddb2d89f8beda66dcecb6b0e0fec30a0b5720a)

BibTeX

@misc{dc7ce1ffa67750,
  title = {Towards Multi-Value Alignment via Value Localization in LLMs — arXiv:2611.02503},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.02503},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7ce1ffa67750; sha256 84b05dd148ad8097f66e936799ddb2d89f8beda66dcecb6b0e0fec30a0b5720a},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02503. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Aligning large language models (LLMs) with multiple, potentially conflicting human values remains challenging, as existing multi-objective alignment methods often require costly fine-tuning and provide limited insight into the internal mechanisms underlying value-related behaviors. In this work, we investigate multi-value alignment from a mechanistic interpretability perspective by identifying value-sensitive attention heads (value heads) in LLMs. Using a probing-based value localization framework, we discover sparse sets of attention heads associated with diverse dimensions, spanning both human values (e.g., harmlessness, honesty) and stylistic preferences (e.g., coherence, complexity). Analyses across multiple LLM families reveal functional specialization and structured interactions among value heads, including conflicts between certain values. Targeted head ablations further induce substantial, value-specific behavioral changes, providing causal evidence of their functional importance. Building on these findings, we propose an inference-time activation steering strategy that enables controllable alignment with individual values or combinations of values without updating the underlying model parameters, supported by theoretical analysis. Experiments across multiple value alignment benchmarks demonstrate effective single- and multi-value control while largely preserving non-target model performance. Our findings connect mechanistic interpretability with flexible multi-value alignment, offering a foundation for more interpretable and pluralistic LLM behavior. The code is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.02503 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:59:45.823Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).