arXiv AI
Aug 20

Position: AI Leaderboards Are Underserving the Global South: A Case Study from India

The paper argues that AI leaderboards are ill-suited for the Global South because they lack independent governance, conflict‑of‑interest policies, and mechanisms for metric evolution. It highlights that high‑quality regional benchmarks already exist—IndicSUPERB, MILU, and LAHAJA for India; IrokoBench for Africa; AlGhafa for Arabic—but are excluded from global leaderboards due to institutional design. A consultation with 58 Indian AI practitioners shows a clear preference for formal governance and disclosure‑based conflict management, leading the authors to propose regional leaderboards with independent governance as the solution.

By Sourav Banerjee, Saikat Saha