Metraipolis › archive › dc_0ea44893403f

LLMs Cannot Be Neutral: Gender Sensitivity Across Candidates, Evaluators, and Models — arXiv:2611.02692

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.02692
License (as archived): CC0-1.0
Archived: 2026-11-03T18:23:43.840Z
SHA-256 of the archived text: ea13a68f3e7a5668696e7c5a06f7a7c0203893ae54f4fe21daf2574a49a7e481
Stable URL of this page: https://metraipolis.com/documento/dc_0ea44893403f

How to cite

LLMs Cannot Be Neutral: Gender Sensitivity Across Candidates, Evaluators, and Models — arXiv:2611.02692 — arXiv (en), https://arxiv.org/abs/2611.02692 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0ea44893403f (Metraipolis archive dc_0ea44893403f, sha256 ea13a68f3e7a5668696e7c5a06f7a7c0203893ae54f4fe21daf2574a49a7e481)

BibTeX

@misc{dc0ea44893403f,
  title = {LLMs Cannot Be Neutral: Gender Sensitivity Across Candidates, Evaluators, and Models — arXiv:2611.02692},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.02692},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0ea44893403f; sha256 ea13a68f3e7a5668696e7c5a06f7a7c0203893ae54f4fe21daf2574a49a7e481},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02692. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) are increasingly used to evaluate candidates for professional roles, raising concerns about their sensitivity to demographic identities. We examine how candidate and evaluator gender assignments affect decisions by LLM-as-a-Judge (LaJ) models and decision language models (DLMs). In a hypothetical city council voting task, eight models evaluated 240 synthetic professional narratives from NVIDIA's US and French Nemotron-Personas datasets under independently varied gender assignments, producing 30,720 judgments. Explicit candidate gender labels increased Yes-vote probability by 1.05-1.73 percentage points relative to unspecified gender, while only the nonbinary evaluator assignment showed a significant increase (1.26 points). Identity matching showed no significant overall association. Yes-vote rates varied across models (81.3-97.2%), and 2.5-3.3% of decisions reversed relative to the unspecified-identity baseline. Under direct substitutions between explicit gender identities, DLMs exhibited fewer decision reversals than LaJ models (1.18% vs. 3.38%), indicating lower observed decision-level gender sensitivity among the tested DLMs. These findings demonstrate that gender sensitivity depends on candidate and evaluator identities and the evaluation model, while highlighting the importance of examining individual decision consistency alongside aggregate differences.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Human-Computer Interaction (cs.HC).
Pagina del lavoro: https://arxiv.org/abs/2611.02692 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T18:10:07.976Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).