arXiv:2606. 16723v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly take actions (screening applicants, recommending credit, triaging patients), yet fairness for LLMs is still measured by grading answers.
By Triveni Morla, Rohith Reddy Bellibaltu, Manpreet Singh, Manmeet Singh Kapoor
arXiv:2507. 11548v3 Announce Type: replace-cross Abstract: The use of publicly available generative AI systems for resume evaluation is often justified by the assumption that these tools reduce bias relative to human judgment.
By Kevin T Webster
PopResume is a population‑representative resume dataset designed for causal fairness auditing of large language model (LLM) and vision‑language model (VLM) resume screeners. It grounds fairness evaluation in real population statistics and preserves natural attribute relationships, enabling path‑specific effect (PSE) analysis that separates business‑necessity from redlining pathways. Using PopResume, the authors evaluated eight models on 60.8K resumes across five occupations and uncovered five discrimination patterns that aggregate metrics missed, demonstrating the value of causally‑grounded auditing.
By Sumin Yu, Juhyeon Park, Taesup Moon
The study evaluates ten bias audit instruments across ten advanced language models on occupational gender, age, and socioeconomic status. While each tool reliably detects bias, their rankings of model performance are essentially random, indicating that different audits measure distinct constructs. The findings show that a single audit can identify bias direction within its own framework, but no audit can consistently rank models against one another.
By William Guey, Pierrick Bougault, Wei Zhang, Vitor D. de Moura, Jos\'e O. Gomes
The paper reviews the evolution of AI recruitment systems from simple profile matching to complex, multi‑stage workflows that retrieve evidence, compare candidates, and execute actions. It analyzes 40 representative works, highlighting transitions from similarity to reciprocal suitability, from single models to compound workflows, and from offline predictions to evidence‑aligned evaluation. The authors identify persistent gaps—such as confounded behavioral labels, limited data validity, and lack of privacy assessment—and propose a staged mapping for defensible evaluation and an agenda for auditable, evidence‑grounded systems.
By Ziyi Zhao, Guanzheng Wei
The paper reports that counterfactual fairness audits of clinical language‑model agents are unreliable without accounting for a per‑action instability floor. By repeatedly running identical vignettes, the authors found that actions changed 8.7% of the time, with instability varying eightfold across actions. A second model confirmed a pooled floor of 6.7%, showing that any reported fairness estimate lacking this floor cannot be interpreted as evidence of disparity.
By Rohith Reddy Bellibaltu, Manpreet Singh, Deepak Parashar, Rahul Joshi