The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Despite groundbreaking advancements in generative models during the last decade, concerns about their lack of fairness, reinforcing societal inequalities and harming marginalized groups, remain under-...
The paper argues that fairness failures in generative models arise mainly from inadequate evaluation practices, making fairness findings hard to compare or use for deployment. It diagnoses common empirical and conceptual shortcomings in current methods and calls for a move toward standardized, generative‑specific evaluation. The authors introduce Fairness Cards, a minimal reporting artifact that explicitly documents evaluation choices—such as prompt families, counterfactual protocols, metrics, and refusal handling—to improve reproducibility, comparability, and accountability.
The paper introduces FAPE, a four‑stage framework for evaluating the post‑processing fairness intervention ThresholdOptimizer across eight diverse domains, including criminal justice, finance, healthcare, and education. It reports that the intervention reduces disparity in most high‑disparity cases but can worsen fairness when baseline disparities are low, and that a single deployment‑time audit is unreliable without continuous monitoring and baseline‑disparity screening.
The paper argues that traditional probabilistic fairness metrics can miss significant disparities in the actual consequences of decisions. By introducing a utility-based framework, the authors show that a process can satisfy ε-fairness yet still be maximally unfair when utilities are considered. They apply this framework to college admissions and credit‑risk assessment, demonstrating that equalizing probabilities alone may mask unequal utility outcomes across groups.
arXiv:2407. 14766v4 Announce Type: replace-cross Abstract: This paper presents a philosophical and experimental study of fairness interventions in AI classification, centered on the explainability and transparency of corrective methods, and on the opposition between two fairness criteria, namely Demographic Parity and Equalized Odds.
arXiv:2607. 17679v1 Announce Type: cross Abstract: This is an invited commentary on the Psychometrika focus article "Fairness Issues and Evaluation in Psychometrics and AI/ML: What Can We Learn from Each Field?