arXiv AI By Mohit Sharma, Okan Ko\c{c}, Amit Deshpande, Takashi Ishida, Gang Niu, Rajiv Ratn Shah, Masashi Sugiyama

Practical Estimation of the Bayes Optimal Fairness-Accuracy Tradeoff with Soft Labels

Read the original on arXiv AI →

The paper presents a method to estimate the Bayes optimal fairness‑accuracy tradeoff using only soft labels and group information. It derives an explicit expression for the fair Bayes error rate and constructs consistent, instance‑free estimators with provable bias bounds and sample complexity guarantees. Empirical results show that the estimators provide a robust, low‑variance estimate of the optimal fairness‑accuracy Pareto frontier even when soft labels are noisy.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Statistics ML
Oct 28

Targeted Learning for Inference on Data Fairness

The paper introduces a targeted learning approach to assess fairness in the data generating process used for training predictive models, rather than focusing solely on model performance. It derives estimators for demographic parity, equal opportunity, and conditional mutual information, and demonstrates through simulations that these estimators improve confidence‑interval coverage and reduce false detections of population unfairness compared to fixed‑model inference. The authors also apply the method to real data to illustrate its practical utility.

By Joseph H. Rudoler, Alexander Asemota, Giles Hooker
arXiv Machine Learning
Sep 3

Bayes-Optimal BER and AUC: Estimation and Evaluation of Estimators

The paper introduces soft‑label‑based estimators for the Bayes‑optimal balanced error rate (BER) and area under the ROC curve (AUC), extending from a clean setting with known class priors to a realistic scenario with unknown priors and corrupted soft labels. It also adapts the FeeBee evaluation framework to assess these estimators without needing the true optimum, providing practical evaluation scores for any estimator of optimal BER or AUC. Experiments on synthetic and real datasets confirm the effectiveness of both the estimators and the evaluation method.

By Ryota Ushio, Takashi Ishida, Masashi Sugiyama