PopResume is a population‑representative resume dataset designed for causal fairness auditing of large language model (LLM) and vision‑language model (VLM) resume screeners. It grounds fairness evaluation in real population statistics and preserves natural attribute relationships, enabling path‑specific effect (PSE) analysis that separates business‑necessity from redlining pathways. Using PopResume, the authors evaluated eight models on 60.8K resumes across five occupations and uncovered five discrimination patterns that aggregate metrics missed, demonstrating the value of causally‑grounded auditing.
By Sumin Yu, Juhyeon Park, Taesup Moon
The paper proposes Causal Evidentiary Governance (CEG), a framework that requires regulated institutions to maintain a versioned directed acyclic graph (DAG) separating allowable from disallowed causal pathways in high‑risk machine learning systems. CEG introduces the Causal Harm Rate to quantify prediction variation due to disallowed pathways and pairs each decision with a signed Decision‑Evidence Packet (DEP) that cryptographically links the prediction to the DAG and path‑specific attributions, enabling efficient inclusion proofs via a Merkle tree. Empirical validation on synthetic credit data and the German Credit dataset demonstrates that CEG more clearly isolates causal effects than traditional fairness metrics and that a proof‑of‑concept implementation shows operational feasibility with manageable performance tradeoffs.
By Samah Kareem, Bar{\i}\c{s} \c{C}elikta\c{s}
arXiv:2601.03087v2 Announce Type: replace
Abstract: Large Language Models (LLMs) exhibit systematic biases across demographic groups. Auditing is proposed as an accountability tool for black-box LLM...
By David Hartmann, Lena Pohlmann, Lelia Hanslik, Noah Gie{\ss}ing, Bettina Berendt, Pieter Delobelle
arXiv:2506. 23033v5 Announce Type: replace Abstract: Fairness audits are a key component of responsible machine-learning deployment.
By Yash Vardhan Tomar
arXiv:2606. 16723v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly take actions (screening applicants, recommending credit, triaging patients), yet fairness for LLMs is still measured by grading answers.
By Triveni Morla, Rohith Reddy Bellibaltu, Manpreet Singh, Manmeet Singh Kapoor
The paper proposes a scalable, automated method for auditing candidate‑job matching systems for demographic bias. It employs large‑language‑model agents to generate neutral resumes, injects controlled demographic variations, ranks candidates with a fine‑tuned embedding model, and evaluates nine fairness metrics across counterfactual, group‑fairness, and merit‑aware families, producing a composite risk report. Experiments on a small corpus show that single‑score audits miss nuanced issues, underscoring the need for multi‑metric evaluation and LLM‑generated audits as a low‑cost complement to human reviews.
By Sai Yashwant, Shruti Bansal, Anurag Dubey, Samaroha Chatterjee, Satyam Kumar, Shreyash Gupta, Gantala Thulsiram
arXiv:2607. 08953v1 Announce Type: new Abstract: Algorithmic fairness methods are increasingly used to identify and mitigate bias in machine learning models, yet most approaches are evaluated in isolation and along single demographic axes.
By Nick Souligne, Isabella Mixton-Garcia, Vignesh Subbian
arXiv:2607. 28608v1 Announce Type: new Abstract: Clinical risk models routinely achieve strong aggregate performance while producing materially different error rates across patient subgroups.
By Sparsh Roy, Samuel Girmachew, Nishita Chavan
arXiv:2607. 28934v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly involved in the distribution of scarce resources, raising concerns about biased allocations based on characteristics like race and gender.
By Martin Lukk (University of Toronto)
The paper introduces REMI, a framework that treats counterfactual fairness as a relational invariant discovery problem. By learning over paired examples, REMI identifies input regions where fairness is violated and generates interpretable rule-based models—fairness invariants—that can block or relabel unfair predictions without retraining the underlying model. Experiments on symbolic and neural network programs show REMI localizes fairness bugs in over 83% of cases and reduces discriminatory decisions in black-box models by up to 70%.
By Ranit Debnath Akash, Ashish Kumar, Gang Tan, Saeid Tizpaz-Niari
arXiv:2609.16321v1 Announce Type: cross
Abstract: Existing fairness analysis tools predominantly operate as post-training evaluation frameworks, requiring practitioners to complete the full model dev...
By Archit Rathod, Saeid Tizpaz-Niari
The paper introduces a causal mediation framework to separate direct discrimination from structural inequality in AI-driven credit decisions. Using Pearl’s natural direct and indirect effects, it presents an identification strategy under treatment‑induced confounding and proposes a doubly‑robust estimator with efficiency guarantees. Empirical analysis of 89,465 mortgage applications shows that about 77% of racial denial disparities stem from financial mediators, while the remaining 23% represents a conservative lower bound on direct discrimination.
By Duraimurugan Rajamanickam