← Back to all news
arXiv AI By Jacob T. Emmerson, Phuong-Anh Nguyen-Le, Ronan Romano, Wilber Sean V. Anterola, Yann Billeter, Zhijing Jin

An Open Pipeline and Dashboard for Systemic-Risk Evidence under the EU AI Act's Code of Practice

Read the original on arXiv AI →

The paper introduces the Systemic Risk Index, an open pipeline and dashboard that aggregates evidence from 19 public AI benchmarks into four systemic‑risk categories defined by the EU GPAI Code of Practice. It evaluates 18 models using harm‑preserving perturbations and simulated deployment contexts, offering users the ability to switch between average and worst‑case aggregation and to trace each risk rating back to its benchmark evidence. The study finds that worst‑case scores can be 14 to 37 points lower than average scores, and that LLM judges agree with human graders at a level comparable to human‑human agreement.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Sep 10

A Translational Note on AI Safety Evaluation

The article discusses how automated red‑teaming can uncover more vulnerabilities at lower cost than human red‑teaming on AI safety benchmarks, yet this comparison conflates measurement with conclusion. It argues that benchmarks only assess harms within a predefined set, leaving a "threat‑model coverage gap" that can hide new risks, as seen in non‑English prompts. The authors suggest that evaluators from deployment contexts distinct from developers are needed to close this gap.

By Madhava Gaikwad
benchmarkssafety
More like this →
arXiv AI
Sep 18

Governance-as-Code: Translating EU AI Act Technical Requirements into Executable Compliance Pipelines for Generative AI Systems

The paper introduces Governance-as-Code (GaC), a framework that translates the EU AI Act’s technical requirements into 43 machine‑checkable acceptance criteria across six compliance modules. GaC runs within a CI/CD pipeline, producing Article‑indexed audit evidence and providing actual Rego policy code. The authors validate GaC on two enterprise deployments, showing it reproduces manual audit findings—including three penalty‑triggering violations—while reducing audit labor by about 75%.

By Rudrendu Kumar Paul, Sourav Nandy
benchmarkssafety
More like this →