Metraipolis › archive › dc_db776228dc9b

Applications of Risk Science to AI Fairness Evaluation: Principles, Challenges, and Best Practices — arXiv:2608.29478

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.29478
License (as archived)
CC0-1.0
Archived
2026-10-17T07:47:49.525Z
SHA-256 of the archived text
e9844c1a9f30d14f16b19272f97f3047cd7373369ca828a3e75a72a823e54590
Stable URL of this page
https://metraipolis.com/documento/dc_db776228dc9b

How to cite

Applications of Risk Science to AI Fairness Evaluation: Principles, Challenges, and Best Practices — arXiv:2608.29478 — arXiv (en), https://arxiv.org/abs/2608.29478 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_db776228dc9b (Metraipolis archive dc_db776228dc9b, sha256 e9844c1a9f30d14f16b19272f97f3047cd7373369ca828a3e75a72a823e54590)
BibTeX
@misc{dcdb776228dc9b,
  title = {Applications of Risk Science to AI Fairness Evaluation: Principles, Challenges, and Best Practices — arXiv:2608.29478},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.29478},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_db776228dc9b; sha256 e9844c1a9f30d14f16b19272f97f3047cd7373369ca828a3e75a72a823e54590},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.29478. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Scholarly work which aims to describe potential societal impacts (e.g., risks) of proliferating technology (especially related to artificial intelligence or other algorithmic systems) is likely to have an impact beyond the scientific communities it was written for, given that general society itself is a primary object of study. However, it is an open question whether the current practices of AI evaluation scholarship follow the principles and best practices established by risk science, which aims to systematically generate knowledge related to understanding, assessing, communicating, managing, and governing risk. In this work, we examine this in depth by conducting a literature review of scholarly works purporting to evaluate the bias or fairness of technological systems used for tasks related to hiring and employment. Through analysis of 22 common fairness evaluation metrics and studies using them, we find that most characterize the severity of bias- or fairness-related consequences but do not follow best practices to characterize the uncertainty around either the occurrence of these consequences or severity estimates. Next, we conduct a case study of fairness evaluation for an AI-mediated resume screening task and demonstrate how principles of risk science can be incorporated into such an evaluation. Finally, we propose the AI Risk Report Card, which facilitates the reporting and communication of risk assessment results to stakeholders in positions to act based on the predicted risks. The outcomes of these activities suggest that further research at the convergence of risk science and AI evaluation can lead to advancements in AI assessments of societal impact by enabling shared frameworks to evaluate and discuss AI risks both within and outside of the scientific community.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.AI).
Categorie arXiv: Computers and Society (cs.CY) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.29478 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T07:28:26.769Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).