Metraipolis › archive › dc_feca36df3bbb

Aligning Language Models Towards Counterfactual Invariance — arXiv:2610.22942

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.22942
License (as archived): CC0-1.0
Archived: 2026-10-19T08:43:04.404Z
SHA-256 of the archived text: c1dad5b623979ca51424158617b78424b4b8fc9753cf9f1ffb9fa8b742c867da
Stable URL of this page: https://metraipolis.com/documento/dc_feca36df3bbb

How to cite

Aligning Language Models Towards Counterfactual Invariance — arXiv:2610.22942 — arXiv (en), https://arxiv.org/abs/2610.22942 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_feca36df3bbb (Metraipolis archive dc_feca36df3bbb, sha256 c1dad5b623979ca51424158617b78424b4b8fc9753cf9f1ffb9fa8b742c867da)

BibTeX

@misc{dcfeca36df3bbb,
  title = {Aligning Language Models Towards Counterfactual Invariance — arXiv:2610.22942},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.22942},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_feca36df3bbb; sha256 c1dad5b623979ca51424158617b78424b4b8fc9753cf9f1ffb9fa8b742c867da},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.22942. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Small variations in prompt characteristics can meaningfully alter the semantics, tone, and contextual assumptions of large language model outputs. Such characteristics can involve group variables such as demographics or other sensitive latent attributes. Sensitivity to such input variables is undesirable when demographic information is causally irrelevant to the task, but might still be relevant for generation of semantically coherent responses. This makes the task of enforcing counterfactual invariance while preserving model utility difficult. To study this problem, we first show that standard model alignment recipes (such as reinforcement learning) increase differences in response quality across demographics, while improving overall response quality, ultimately amplifying what is known as counterfactual bias. To address this, we develop a policy gradient technique involving a group-level reward function that operates on matched pairs of counterfactual prompts. For completeness, we build a dataset of controlled counterfactual prompt pairs and design an evaluation that separately measures cases where demographic information is causally relevant to the answer. Finally, we validate our findings on several fairness benchmarks. We demonstrate that our methods reduce counterfactual differences while preserving overall response quality and that the models continue to adapt appropriately when such sensitive information is relevant.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.22942 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T08:39:10.135Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).