arXiv Machine Learning

Fairness Theatre: Evaluating Post-Hoc Fairness Interventions in Vendor-Controlled Early Warning Systems

arXiv AI
Aug 19

Position: Fairness Failure in Generative Models is an Evaluation Problem

The paper argues that fairness failures in generative models arise mainly from inadequate evaluation practices, making fairness findings hard to compare or use for deployment. It diagnoses common empirical and conceptual shortcomings in current methods and calls for a move toward standardized, generative‑specific evaluation. The authors introduce Fairness Cards, a minimal reporting artifact that explicitly documents evaluation choices—such as prompt families, counterfactual protocols, metrics, and refusal handling—to improve reproducibility, comparability, and accountability.

By Mariia Vladimirova, Jean-Yves Franceschi, Thibaut Issenhuth
arXiv Machine Learning
Sep 24

When Post-Processing Fairness Constraints Help and When They Harm: Evidence from Eight Cross-Domain Evaluations

The paper introduces FAPE, a four‑stage framework for evaluating the post‑processing fairness intervention ThresholdOptimizer across eight diverse domains, including criminal justice, finance, healthcare, and education. It reports that the intervention reduces disparity in most high‑disparity cases but can worsen fairness when baseline disparities are low, and that a single deployment‑time audit is unreliable without continuous monitoring and baseline‑disparity screening.

By Nithin Raghava Ramachandra Narla
arXiv Machine Learning
Sep 18

When fairness metrics fail: A utility-based perspective on $\varepsilon$-fairness

The paper argues that traditional probabilistic fairness metrics can miss significant disparities in the actual consequences of decisions. By introducing a utility-based framework, the authors show that a process can satisfy ε-fairness yet still be maximally unfair when utilities are considered. They apply this framework to college admissions and credit‑risk assessment, demonstrating that equalizing probabilities alone may mask unequal utility outcomes across groups.

By Tolulope Fadina, Thorsten Schmidt
arXiv Machine Learning
Jun 2

Beyond Procedure: Substantive Fairness in Conformal Prediction

arXiv:2602. 16794v2 Announce Type: replace-cross Abstract: Conformal prediction (CP) offers distribution-free uncertainty quantification for machine learning models, yet its interplay with fairness in downstream decision-making remains underexplored.

By Pengqi Liu, Zijun Yu, Mouloud Belbahri, Arthur Charpentier, Masoud Asgharian, Jesse C. Cresswell
arXiv AI
Sep 18

The AR Fairness Metamodel: A Structured Framework for Fairness Measures

The paper introduces the AR Fairness Metamodel, a structured framework for representing, analyzing, and comparing fairness scenarios. It incorporates key elements such as agents, resources, and their attributes, and supports both discrete and continuous fairness measures—including equality, equity, group fairness, individual fairness, the Gini index, the Theil index, Jain's fairness index, and a specific measure for Australia's Child Care Subsidy. The metamodel builds on the Tiles framework, offering modular components that can be connected to capture diverse fairness definitions, and includes formal proofs of relationships among group fairness, individual fairness, and envy‑freeness. An open‑source implementation of the Tiles framework is provided to facilitate practical fairness modeling and evaluation across various applications.

By Julian Alfredo Mendez, Timotheus Kampik
arXiv Machine Learning
Sep 15

From Network Inequality to Network Fairness: A Perspective on Responsible Decision-Making

The article discusses how social networks influence decision-making and opportunity distribution, noting that network-generating mechanisms often mirror existing inequalities and can amplify disparities when used in technology. It identifies ten network effects that bias the link between intended measurements and observed data, using academic hiring as a case study to show that network biases can be neither inherently harmful nor beneficial. The authors argue for a comprehensive, networked fairness framework that incorporates both distributive and procedural justice and involves all stakeholders.

By Lisette Esp\'in-Noboa, Tina Eliassi-Rad, Pak-Hang Wong, Erich Prem, Meike Zehlike, Ricardo Baeza-Yates, Suresh Venkatasubramanian, Fariba Karimi