← Back to all news
arXiv Machine Learning By Xiaotong Wang, Xuan Xie

FairTest: Search-Based Fairness Testing for Multi-Agent Reinforcement Learning Systems

Read the original on arXiv Machine Learning →

FairTest is a search-based testing framework designed to uncover fairness failures in Multi-Agent Reinforcement Learning (MARL) systems. It guides candidate generation using three fitness functions—measuring observed fairness, predicting fairness from abstract states, and assessing policy decision uncertainty—and prioritizes tests based on predicted fairness and uncertainty. Evaluations on three environments and two MARL algorithms show FairTest detects significantly more fairness failures than three baselines, with a 221% increase in failure count and 23% better coverage on average.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Aug 25

Learning from the Test: Self-Referential Differential Testing for Deep RL Agents

The paper introduces Delta, a two‑phase framework for testing deep reinforcement learning agents. In the first phase, the agent under test is evaluated for catastrophic failures while collecting decision‑making data. The second phase trains a challenger agent from this data using offline RL; comparing the challenger’s rewards to the original agent reveals optimality bugs, and Delta successfully uncovered thousands of such issues across multiple environments.

By Junda He, Jieke Shi, Zhou Yang, Mingfei Cheng, David Lo
agentsreinforcement-learningsafety
More like this →
arXiv AI
Jun 2

LLM4Cov: Execution-Aware Agentic Learning for High-coverage Testbench Generation

arXiv:2602. 16953v3 Announce Type: replace Abstract: Execution-aware LLM agents offer a promising paradigm for learning from tool feedback, but such feedback can be expensive and slow to obtain, making online reinforcement learning (RL) less practical in certain scenarios.

By Hejia Zhang, Zhongming Yu, Chia-Tung Ho, Haoxing Ren, Brucek Khailany, Jishen Zhao
llmsagentsreinforcement-learningbenchmarks
More like this →
arXiv AI
Sep 15

Evaluation Metrics for Safe Reinforcement Learning

The paper introduces new evaluation metrics for safe reinforcement learning that go beyond average safety guarantees by examining how often and how severely safety bounds are violated, consistency across tasks and bounds, and the relationship between training-time and final policy behavior. It also proposes a safety tier system for categorizing algorithms and presents empirical safety evaluations on multiple navigation tasks. The authors recommend reporting aggregate metrics, distributional data, and task‑specific results together, and provide an open‑source suite, SafeRLEval, to facilitate reliable safety assessment.

By Lindsay Spoor, Aske Plaat, Thomas Moerland
agentsreinforcement-learningbenchmarkssafety
More like this →