Position: Fairness Failure in Generative Models is an Evaluation Problem
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
The paper argues that fairness failures in generative models arise mainly from inadequate evaluation practices, making fairness findings hard to compare or use for deployment. It diagnoses common empirical and conceptual shortcomings in current methods and calls for a move toward standardized, generative‑specific evaluation. The authors introduce Fairness Cards, a minimal reporting artifact that explicitly documents evaluation choices—such as prompt families, counterfactual protocols, metrics, and refusal handling—to improve reproducibility, comparability, and accountability.
arXiv:2608.02551v2 Announce Type: replace-cross Abstract: Fairness evaluation concerns not only what a model produces, but also what its outputs ought to be compared against. When a model generates "...
arXiv:2407. 14766v4 Announce Type: replace-cross Abstract: This paper presents a philosophical and experimental study of fairness interventions in AI classification, centered on the explainability and transparency of corrective methods, and on the opposition between two fairness criteria, namely Demographic Parity and Equalized Odds.
arXiv:2606. 26200v1 Announce Type: cross Abstract: Modern machine learning systems have outgrown their origins as isolated predictive constructs, evolving into complex socio-technical architectures that actively mediate human opportunity.
The paper examines how public institutions, particularly a Canadian college, can address inequities in proprietary Early Warning Systems (EWS) when they cannot modify the underlying AI models. By testing six post‑hoc fairness interventions on a research EWS under realistic procurement constraints, the authors compare fairness, accuracy, and demographic disparities, and introduce error‑type profiling to track how interventions shift false positives and false negatives. The results show that while interventions can redistribute disparities, they do not consistently reduce them; some favor already‑advantaged groups, leaving small, marginalized groups poorly served, illustrating the limits of fairness work under vendor‑controlled systems.
arXiv:2602. 16794v2 Announce Type: replace-cross Abstract: Conformal prediction (CP) offers distribution-free uncertainty quantification for machine learning models, yet its interplay with fairness in downstream decision-making remains underexplored.