arXiv AI

The AR Fairness Metamodel: A Structured Framework for Fairness Measures

The paper introduces the AR Fairness Metamodel, a structured framework for representing, analyzing, and comparing fairness scenarios. It incorporates key elements such as agents, resources, and their attributes, and supports both discrete and continuous fairness measures—including equality, equity, group fairness, individual fairness, the Gini index, the Theil index, Jain's fairness index, and a specific measure for Australia's Child Care Subsidy. The metamodel builds on the Tiles framework, offering modular components that can be connected to capture diverse fairness definitions, and includes formal proofs of relationships among group fairness, individual fairness, and envy‑freeness. An open‑source implementation of the Tiles framework is provided to facilitate practical fairness modeling and evaluation across various applications.

arXiv AI
Aug 19

Position: Fairness Failure in Generative Models is an Evaluation Problem

The paper argues that fairness failures in generative models arise mainly from inadequate evaluation practices, making fairness findings hard to compare or use for deployment. It diagnoses common empirical and conceptual shortcomings in current methods and calls for a move toward standardized, generative‑specific evaluation. The authors introduce Fairness Cards, a minimal reporting artifact that explicitly documents evaluation choices—such as prompt families, counterfactual protocols, metrics, and refusal handling—to improve reproducibility, comparability, and accountability.

By Mariia Vladimirova, Jean-Yves Franceschi, Thibaut Issenhuth
arXiv Machine Learning
Sep 18

When fairness metrics fail: A utility-based perspective on $\varepsilon$-fairness

The paper argues that traditional probabilistic fairness metrics can miss significant disparities in the actual consequences of decisions. By introducing a utility-based framework, the authors show that a process can satisfy ε-fairness yet still be maximally unfair when utilities are considered. They apply this framework to college admissions and credit‑risk assessment, demonstrating that equalizing probabilities alone may mask unequal utility outcomes across groups.

By Tolulope Fadina, Thorsten Schmidt
arXiv Machine Learning
Sep 21

FairLMs: A Turnkey Library for Fairness in Language Models

FairLMs is a Python library designed to streamline fairness research in language models by unifying bias measurement, mitigation, and evaluation evidence. It offers 33 intrinsic and extrinsic metrics, 14 mitigation components across four intervention categories, 14 diagnostic tools, adapters for major Transformer architectures and hosted APIs, and benchmark loaders. The library enforces explicit declarations of model capabilities and input requirements, ensuring compatibility and reproducibility across components and datasets.

By Jiale Zhang, Michael Larionov, Zichong Wang, Zhipeng Yin, Wenbin Zhang
arXiv AI
Oct 28

Fairness Definitions in Language Models Explained

The paper "Fairness Definitions in Language Models Explained" surveys and clarifies various fairness notions applicable to language models (LMs). It introduces a taxonomy categorizing these concepts by transformer architecture—encoder-only, decoder-only, and encoder-decoder—and demonstrates each definition through experiments to illustrate practical implications. The authors also discuss current research challenges and open questions to encourage further innovation in the field.

By Zhipeng Yin, Zichong Wang, Avash Palikhe, Wenbin Zhang
arXiv AI
Sep 25

Fair Like Us? Auditing LLM Alignment in Resource Allocation

The paper "Fair Like Us? Auditing LLM Alignment in Resource Allocation" presents a method for evaluating how large language models reason about fairness in the allocation of scarce, indivisible resources. It compares LLMs’ first‑person fairness judgments with human responses across various scenarios, finding that models tend to favor stricter fairness constraints, exhibit more self‑interested behavior, and are sensitive to framing. The study also shows that current fine‑tuning datasets struggle to align LLM judgments with human ones.

By Qishen Han, Hadi Hosseini, Joshua Kavner, Samarth Khanna, Sujoy Sikdar, Lirong Xia
arXiv Machine Learning
Oct 2

FairRARI: A Plug and Play Framework for Fairness-Aware PageRank

FairRARI is a new in‑processing convex optimization framework that computes PageRank vectors while satisfying various group‑fairness constraints based on vertex attributes. By formulating PageRank as a variational problem, FairRARI solves a strongly convex problem that guarantees a target fairness level and retains the same asymptotic time complexity as standard PageRank. Experiments on real datasets show that FairRARI outperforms existing methods in utility while achieving the desired fairness across multiple vertex groups.

By Emmanouil Kariotakis, Aritra Konar