arXiv Machine Learning

Audit Me If You Can: Query-Efficient Active Fairness Auditing of Black-Box LLMs

arXiv Machine Learning
Oct 8

Robust Decentralized Fairness Auditing

arXiv:2610.10199v1 Announce Type: new Abstract: Emerging legislation requires large language models (LLMs) to be audited for compliance with regulatory standards, particularly fairness. Such black-bo...

By Sayan Biswas, Jade Garcia Bourr\'ee, Anne-Marie Kermarrec, Palak, Martijn de Vos
arXiv Machine Learning
Sep 18

Accuracy Is Not Enough: A Cross-Architecture Audit of Demographic Bias in Deep Knowledge Tracing

The study audits demographic bias across four deep knowledge tracing architectures—DKT, DKVMN, SAKT, and AKT—using two large public datasets (Eedi and OULAD). It finds that bias is context‑dependent: socioeconomic bias is significant on Eedi, while gender bias appears on OULAD for most models. The most accurate model, AKT, also exhibits the greatest bias, and standard mitigation techniques such as reweighting and adversarial debiasing fail to reduce bias without sacrificing accuracy.

By Dang Quang Minh, Nguyen Dung Son, Nguyen Huu Loi, Truong Viet Vu, Nguyen Thai Anh
arXiv Machine Learning
Sep 24

When Post-Processing Fairness Constraints Help and When They Harm: Evidence from Eight Cross-Domain Evaluations

The paper introduces FAPE, a four‑stage framework for evaluating the post‑processing fairness intervention ThresholdOptimizer across eight diverse domains, including criminal justice, finance, healthcare, and education. It reports that the intervention reduces disparity in most high‑disparity cases but can worsen fairness when baseline disparities are low, and that a single deployment‑time audit is unreliable without continuous monitoring and baseline‑disparity screening.

By Nithin Raghava Ramachandra Narla
arXiv Computation and Language
Oct 27

From Self-Confidence to Statistical Accuracy: Scaling Human-in-the-Loop Evaluation with Direct and LLM-Assisted Auditing

arXiv:2610.35698v1 Announce Type: new Abstract: For large language models (LLMs) and multimodal large language models (MLLMs), model-reported self-confidence is often miscalibrated, yet it may still...

By Harine Choi (Texas A&M Transportation Institute, Texas A&M University System), Chenchen Kuai (Zachry Department of Civil and Environmental Engineering, Texas A&M University), Zihao Li (Department of Civil, Construction and Environmental Engineering, Marquette University), Jiwan Jiang (Zachry Department of Civil and Environmental Engineering, Texas A&M University), Yunlong Zhang (Zachry Department of Civil and Environmental Engineering, Texas A&M University), Zhiwen Fan (Department of Electrical and Computer Engineering, Texas A&M University), Zhengzhong Tu (Department of Computer Science and Engineering, Texas A&M University), Yang Zhou (Zachry Department of Civil and Environmental Engineering, Texas A&M University)