arXiv AI
Jul 1
Wait, am I Being Fair? Characterizing Deductive Stereotyping and Mitigating It with Fair-GCG
arXiv:2606. 30989v1 Announce Type: cross Abstract: Warning: This paper contains several toxic and offensive statements.
Fair Like Us? Auditing LLM Alignment in Resource Allocation
The paper "Fair Like Us? Auditing LLM Alignment in Resource Allocation" presents a method for evaluating how large language models reason about fairness in the allocation of scarce, indivisible resources. It compares LLMs’ first‑person fairness judgments with human responses across various scenarios, finding that models tend to favor stricter fairness constraints, exhibit more self‑interested behavior, and are sensitive to framing. The study also shows that current fine‑tuning datasets struggle to align LLM judgments with human ones.
Related stories
arXiv AI
Jun 19
DeFrame: Debiasing Large Language Models Against Framing Effects
arXiv:2602. 04306v2 Announce Type: replace-cross Abstract: As large language models (LLMs) are increasingly deployed in real-world applications, ensuring their fair responses across demographics has become crucial.
arXiv Machine Learning
Aug 10
Let's Unlearn Stereotypes Before Decision-Making: Assessing the Impact of Intrinsic Bias Mitigation on Downstream Fairness in LLMs
arXiv:2509. 16462v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are increasingly used in high-stakes decision-making systems, where biased predictions can reinforce social and economic disparities.
arXiv Computation and Language
Sep 2
Evaluating Second-Order Bias of LLMs Through Epistemic Entitlement
The paper introduces a novel framework for assessing second‑order bias in large language models (LLMs), defined as bias in how an LLM judges the acceptability of biased content. Using principles from entitlement epistemology, the authors design a reasoning task that asks LLMs to determine whether a biased text is acceptable for specific demographic groups, and propose two metrics to quantify biased judgments. Experiments on both open‑source and closed‑source models reveal that the task bypasses safety guardrails, uncovers systematic variations across target groups, and demonstrates that models still rely on demographic labels when evaluating bias.
arXiv Computation and Language
Sep 15
Intersectional Fairness in Large Language Models
arXiv:2604.20677v3 Announce Type: replace Abstract: Large Language Models (LLMs) are increasingly deployed in socially sensitive settings, raising concerns about fairness and bias, particularly when...
arXiv AI
Jul 10
Adaptive Generation of Bias-Eliciting Questions for LLMs
arXiv:2510. 12857v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are now widely deployed in user-facing applications, reaching hundreds of millions of users worldwide.
arXiv AI
Aug 3
FairFund-Bench: Evaluating Distributive Bias in LLM Resource Allocation
arXiv:2607. 28934v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly involved in the distribution of scarce resources, raising concerns about biased allocations based on characteristics like race and gender.
arXiv AI
Jul 24
Unlearning Under Imbalance: Benchmarking Fairness in Multimodal LLM Unlearning
arXiv:2607. 21300v1 Announce Type: cross Abstract: Machine unlearning has emerged as a tool for removing personal data from trained models to comply with recent AI regulations.
arXiv AI
23h ago
Efficient Active Auditing of Multi-Group Fairness with Bias Probes
arXiv:2609.40034v1 Announce Type: cross Abstract: Over the past decade, Machine Learning (ML) has been trained under dual objectives: minimizing prediction error via Empirical Risk Minimization (ERM)...
arXiv Machine Learning
Sep 1
Audit Me If You Can: Query-Efficient Active Fairness Auditing of Black-Box LLMs
arXiv:2601.03087v2 Announce Type: replace Abstract: Large Language Models (LLMs) exhibit systematic biases across demographic groups. Auditing is proposed as an accountability tool for black-box LLM...
arXiv AI
Jun 24
When Preferences Fail to Become Incentives: A Utility-Behavior Gap in Large Language Models
arXiv:2606. 22974v2 Announce Type: replace Abstract: Recent work on preference elicitation in large language models (LLMs) has demonstrated that, when given a series of choices between two outcomes, LLMs reveal a coherent, model-specific utility structure.
arXiv AI
1d ago
PADM\'E: Preference Alignment Data Synthesis for Meta-Evaluation of LM Agent Evaluators
PADM'E is a method for synthesizing preference‑aligned data to meta‑evaluate language‑model (LM) evaluators of agentic behaviors. It reframes meta‑evaluation as a preference judgment problem, generating criterion‑based data with small LMs and no human involvement. In a prototype, PADM'E produced 1,000 samples across four domains and three criteria, and human validation showed agreement with human judgment rising from 73% to 85% compared to a naive baseline.