← Back to all news
arXiv AI By Sumaya Nur Adan, Joanna Wiaterek, Nourhan Abdelrhim, Sydney Reis, Robert F. Trager, Benjamin Rosman, Marina Jirotka

Who Can Evaluate AI Independently? A Framework for Analysing Evaluation Infrastructure

Read the original on arXiv AI →

The paper "Who Can Evaluate AI Independently? A Framework for Analysing Evaluation Infrastructure" introduces the PAAI framework—Power, Access, Adequacy, Independence—to assess who controls AI evaluation infrastructure and how that shapes independent evaluation. Applying PAAI to twelve African AI evaluation projects and three joint testing exercises of the AI Safety Institutes, the authors identify three recurring configurations: extractive collaboration, dependent innovation, and constrained independence. The study finds that African actors often provide data and expertise while external institutions retain control over infrastructure and evaluation frameworks, and that participation alone does not confer evaluative authority.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Jul 17

Global Index on Responsible AI: 2026 Report

arXiv:2607. 14782v1 Announce Type: new Abstract: Grounded in human rights-based frameworks such as the UNESCO Recommendation on the Ethics of AI, the Global Index on Responsible AI (GIRAI) examines how countries translate responsible AI commitments into enforceable protections, institutional capacity, and redress mechanisms.

By Rachel Adams, Fola Adeleke, Ayantola Alayande, Selamawit Engida Abdella, Ana Florido, Nicol\'as Grossman, Leah Junck
safety
More like this →
arXiv AI
Aug 18

An Evaluation Framework for National AI Regulation

arXiv:2608. 15417v1 Announce Type: cross Abstract: Governments use laws, institutions, funding programs and nonbinding guidance to shape how AI is developed and used.

By Kaushik Sanjay Prabhakar, Tarun Adarsh R S, Amal Dhivyan Gregory, Sreeparvathy Sajeev, Utkarsh Tomar, Avyay M Casheekar
More like this →
arXiv AI
Aug 20

Global Index on Responsible AI 2026 : Conceptual Framework and Methodology

The Global Index on Responsible AI 2026 (GIRAI) 2nd Edition refines its predecessor by distinguishing between framework existence and implementation, expanding from three to five thematic areas, and adding granular variables for framework quality. It evaluates responsible AI governance across five dimensions—Inclusion and Diversity, Ethics and Sustainability, Labour and Skills, Trust and Safety, and Use of AI in Public Service—using 38 indicators organized into three pillars: AI Policy, CSO Engagement, and Enabling Conditions, plus a separate Use of Unacceptable Risk AI penalty. Data from 135 country-level researchers and secondary sources are normalized to a 100-point scale, weighted by pillar importance, and used to facilitate systematic cross‑national comparisons for policymakers, civil society, and AI developers.

By Fola Adeleke, Rachel Adams, Ayantola Alayande, Daniela Benavente, Ana Florido, Nicol\'as Grossman, Leah Junck
safety
More like this →
arXiv Machine Learning
Sep 11

From Protocols to Evidence: Bounded Claims for AI in Service of the Common Good

The paper argues that AI should be evaluated not only by principles but by concrete protocols that translate commitments into roles, requirements, records, oversight, and assessment. It introduces a rupture test linking institutional baselines to system evaluation, and distinguishes evidence‑bounded deployment from measurement‑bounded governance. The authors propose the RISE AI architecture to make bounded, evidence‑based claims about Responsibility, Inclusivity, Safety, and Empowerment, emphasizing the need for engineering, institutional repair, and ongoing moral judgment.

By Nitesh V. Chawla, Paulo Benanti
safety
More like this →