Metraipolis › archive › dc_a94ebe641de6

Position: AI Leaderboards Are Underserving the Global South: A Case Study from India — arXiv:2608.18117

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.18117
License (as archived)
CC0-1.0
Archived
2026-10-18T13:44:18.811Z
SHA-256 of the archived text
c9448c071cc9f60373336e283e334d731229f39d7176abfa2edfd9b1c5ffe37f
Stable URL of this page
https://metraipolis.com/documento/dc_a94ebe641de6

How to cite

Position: AI Leaderboards Are Underserving the Global South: A Case Study from India — arXiv:2608.18117 — arXiv (en), https://arxiv.org/abs/2608.18117 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a94ebe641de6 (Metraipolis archive dc_a94ebe641de6, sha256 c9448c071cc9f60373336e283e334d731229f39d7176abfa2edfd9b1c5ffe37f)
BibTeX
@misc{dca94ebe641de6,
  title = {Position: AI Leaderboards Are Underserving the Global South: A Case Study from India — arXiv:2608.18117},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.18117},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a94ebe641de6; sha256 c9448c071cc9f60373336e283e334d731229f39d7176abfa2edfd9b1c5ffe37f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.18117. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

This position paper argues that AI leaderboards are structurally ill-suited to serving the Global South because they lack independent governance, conflict-of-interest policies, and mechanisms for metric evolution. The barrier is not missing data; high-quality regional benchmarks already exist: IndicSUPERB, MILU, and LAHAJA for India; IrokoBench for Africa; AlGhafa for Arabic. The barrier is institutional design. Global leaderboards do not include these benchmarks, and no governance mechanism compels them to do so. Commercial pressure corrects leaderboard failures when paying customers in the Global North are affected. The Global South lacks equivalent leverage. Without governance, failures affecting Hindi, Swahili, or Arabic speakers persist indefinitely as documented but unaddressed gaps. Using India as a case study (1.4 billion people, 22 scheduled languages, high-quality benchmarks, but no trusted aggregation), we report findings from a consultation with 58 AI practitioners showing consistent preference for formal governance and disclosure-based conflict management. The solution is not more data but better institutions: regional leaderboards with independent governance from the start.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-20 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; General Economics (econ.GN).
Pagina del lavoro: https://arxiv.org/abs/2608.18117 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T13:41:17.841Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-20?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).