The paper argues that fairness failures in generative models arise mainly from inadequate evaluation practices, making fairness findings hard to compare or use for deployment. It diagnoses common empirical and conceptual shortcomings in current methods and calls for a move toward standardized, generative‑specific evaluation. The authors introduce Fairness Cards, a minimal reporting artifact that explicitly documents evaluation choices—such as prompt families, counterfactual protocols, metrics, and refusal handling—to improve reproducibility, comparability, and accountability.
By Mariia Vladimirova, Jean-Yves Franceschi, Thibaut Issenhuth
arXiv:2608.02551v2 Announce Type: replace-cross
Abstract: Fairness evaluation concerns not only what a model produces, but also what its outputs ought to be compared against. When a model generates "...
By Zeshen Zheng, Yujia He, Qianmian Lin, Xiangyue Huang, Wenqing Chen
arXiv:2407. 14766v4 Announce Type: replace-cross Abstract: This paper presents a philosophical and experimental study of fairness interventions in AI classification, centered on the explainability and transparency of corrective methods, and on the opposition between two fairness criteria, namely Demographic Parity and Equalized Odds.
By Thomas Souverain, Paul \'Egr\'e
arXiv:2606. 26200v1 Announce Type: cross Abstract: Modern machine learning systems have outgrown their origins as isolated predictive constructs, evolving into complex socio-technical architectures that actively mediate human opportunity.
By Antonio Ferrara
arXiv:2602. 16794v2 Announce Type: replace-cross Abstract: Conformal prediction (CP) offers distribution-free uncertainty quantification for machine learning models, yet its interplay with fairness in downstream decision-making remains underexplored.
By Pengqi Liu, Zijun Yu, Mouloud Belbahri, Arthur Charpentier, Masoud Asgharian, Jesse C. Cresswell
arXiv:2509. 16462v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are increasingly used in high-stakes decision-making systems, where biased predictions can reinforce social and economic disparities.
By Mina Arzaghi, Alireza Dehghanpour Farashah, Florian Carichon, Jean-Fran\c{c}ois Plante, Golnoosh Farnadi
arXiv:2607. 17679v1 Announce Type: cross Abstract: This is an invited commentary on the Psychometrika focus article "Fairness Issues and Evaluation in Psychometrics and AI/ML: What Can We Learn from Each Field?
By Youmi Suk
The paper introduces the AR Fairness Metamodel, a structured framework for representing, analyzing, and comparing fairness scenarios. It incorporates key elements such as agents, resources, and their attributes, and supports both discrete and continuous fairness measures—including equality, equity, group fairness, individual fairness, the Gini index, the Theil index, Jain's fairness index, and a specific measure for Australia's Child Care Subsidy. The metamodel builds on the Tiles framework, offering modular components that can be connected to capture diverse fairness definitions, and includes formal proofs of relationships among group fairness, individual fairness, and envy‑freeness. An open‑source implementation of the Tiles framework is provided to facilitate practical fairness modeling and evaluation across various applications.
By Julian Alfredo Mendez, Timotheus Kampik
The paper "Fair Like Us? Auditing LLM Alignment in Resource Allocation" presents a method for evaluating how large language models reason about fairness in the allocation of scarce, indivisible resources. It compares LLMs’ first‑person fairness judgments with human responses across various scenarios, finding that models tend to favor stricter fairness constraints, exhibit more self‑interested behavior, and are sensitive to framing. The study also shows that current fine‑tuning datasets struggle to align LLM judgments with human ones.
By Qishen Han, Hadi Hosseini, Joshua Kavner, Samarth Khanna, Sujoy Sikdar, Lirong Xia
arXiv:2607. 08953v1 Announce Type: new Abstract: Algorithmic fairness methods are increasingly used to identify and mitigate bias in machine learning models, yet most approaches are evaluated in isolation and along single demographic axes.
By Nick Souligne, Isabella Mixton-Garcia, Vignesh Subbian
Algorithmic fairness methods are increasingly used to identify and mitigate bias in machine learning models, yet most approaches are evaluated in isolation and along single demographic axes. This limits practical guidance for selecting fairness strategies, where disparities may arise across intersectional subgroups and across multiple stages of the modeling lifecycle.
FairLMs is a Python library designed to streamline fairness research in language models by unifying bias measurement, mitigation, and evaluation evidence. It offers 33 intrinsic and extrinsic metrics, 14 mitigation components across four intervention categories, 14 diagnostic tools, adapters for major Transformer architectures and hosted APIs, and benchmark loaders. The library enforces explicit declarations of model capabilities and input requirements, ensuring compatibility and reproducibility across components and datasets.
By Jiale Zhang, Michael Larionov, Zichong Wang, Zhipeng Yin, Wenbin Zhang