arXiv Machine Learning By Atsutoshi Kumagai, Tomoharu Iwata, Hiroshi Takahashi, Taishi Nishiyama, Kazuki Adachi, Yasuhiro Fujiwara

AUC Maximization from Biased Positive-unlabeled Data with Confidence

Read the original on arXiv Machine Learning →

The paper introduces a method for maximizing the area under the receiver operating characteristic curve (AUC) when only biased positive and unlabeled (PU) data are available. It leverages confidence scores—probabilities that an instance is positive—associated with a small set of labeled positives to derive an AUC risk estimator that accounts for bias. Experiments on eight real-world datasets demonstrate the method’s effectiveness.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning
Sep 3

Bayes-Optimal BER and AUC: Estimation and Evaluation of Estimators

The paper introduces soft‑label‑based estimators for the Bayes‑optimal balanced error rate (BER) and area under the ROC curve (AUC), extending from a clean setting with known class priors to a realistic scenario with unknown priors and corrupted soft labels. It also adapts the FeeBee evaluation framework to assess these estimators without needing the true optimum, providing practical evaluation scores for any estimator of optimal BER or AUC. Experiments on synthetic and real datasets confirm the effectiveness of both the estimators and the evaluation method.

By Ryota Ushio, Takashi Ishida, Masashi Sugiyama
arXiv Statistics ML
Aug 25

Robust performance metrics for imbalanced classification problems

The paper demonstrates that common binary classification metrics—Matthews' correlation coefficient, Cohen's κ, the F-score, and the Jaccard similarity—are not robust to extreme class imbalance, as the Bayes classifier’s true positive rate tends to zero when the minority class proportion vanishes. To address this, the authors propose robustified versions of these metrics that include a tuning parameter, ensuring that the Bayes-optimal classifier’s threshold remains bounded and its true positive rate stays above zero even in highly imbalanced scenarios. The study provides theoretical bounds, simulation results, and practical guidance on applying these robust metrics to real data, such as a credit‑default dataset, and discusses their relationship to ROC and precision‑recall curves.

By Hajo Holzmann, Bernhard Klar
arXiv Machine Learning
Jun 16

Imbalanced Classification under Capacity Constraints

arXiv:2605. 03289v2 Announce Type: replace-cross Abstract: Detecting observations from a minority class under severe class imbalance is a central challenge in applications such as fraud detection, medical screening, and industrial quality control.

By Daniel Fraiman, Ricardo Fraiman