Metraipolis › archive › dc_f8aebf3accfb

The Audit Decides the Verdict: Instrument Effects Rival Demographic Bias in LLM Decision Audits — arXiv:2609.09048

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.09048
License (as archived): CC0-1.0
Archived: 2026-10-16T12:49:26.013Z
SHA-256 of the archived text: 5cee0b0b09c0af9b22d5774592c07e0a7f66332e12f29dd8119ab8c6cb871351
Stable URL of this page: https://metraipolis.com/documento/dc_f8aebf3accfb

How to cite

The Audit Decides the Verdict: Instrument Effects Rival Demographic Bias in LLM Decision Audits — arXiv:2609.09048 — arXiv (en), https://arxiv.org/abs/2609.09048 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f8aebf3accfb (Metraipolis archive dc_f8aebf3accfb, sha256 5cee0b0b09c0af9b22d5774592c07e0a7f66332e12f29dd8119ab8c6cb871351)

BibTeX

@misc{dcf8aebf3accfb,
  title = {The Audit Decides the Verdict: Instrument Effects Rival Demographic Bias in LLM Decision Audits — arXiv:2609.09048},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.09048},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f8aebf3accfb; sha256 5cee0b0b09c0af9b22d5774592c07e0a7f66332e12f29dd8119ab8c6cb871351},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.09048. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Whether a language model looks demographically biased can depend on how the audit asks its question. A charitable-aid benchmark reports that the same models favor minority applicants when rating requests one at a time and penalize some when ranking side by side. We test whether that reversal generalizes to hiring, lending, and medical triage: 40,726 requests to five models, applications differing only in the applicant's name, and a primary test fixed before collection. It does not. None of 36 planned contrasts survives correction. The rating advantage keeps its sign at roughly half the published size, and a precision extension bounds any hiring ranking penalty below the published effect, though the lending and triage ranking floors sit above that margin, so the exclusion is conclusive for hiring ranking and for rating in all three domains only. Planted disparities tracking their injected sizes and a directional replication on the original aid materials bound these nulls. The audit is livelier than the demographics: models recognize transparent audits nearly always, tie every identical-content comparison whether the varying detail is race or a hobby, and reward first-listed candidates as much as any demographic effect we measure. Audit verdicts reflect audit construction more than demographic bias.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-09 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Computers and Society (cs.CY).
Pagina del lavoro: https://arxiv.org/abs/2609.09048 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T12:30:25.211Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).