OpenAI Blog
Computational limitations in robust classification and win-win results
Read the original on OpenAI Blog →The Flow has not summarised this story yet — read it at OpenAI Blog.
Related stories
arXiv Machine Learning
Jul 10
Robustness Quantification for Discriminative Models: a New Robustness Metric and its Application to Dynamic Classifier Selection
arXiv:2603. 23318v2 Announce Type: replace Abstract: Among the different possible strategies for evaluating the reliability of individual predictions of classifiers, robustness quantification stands out as a method that evaluates how much uncertainty a classifier could cope with before changing its prediction.
arXiv Machine Learning
Sep 11
Local Robustness Quantification for Naive Bayes Classifiers and Generative Forests: a General Approach
The paper introduces techniques for measuring the robustness of predictions made by two generative classifiers—naive Bayes classifiers and generative forests—whose underlying models are probabilistic graphical models. Robustness is defined as the degree to which the classifier’s distribution can be perturbed without altering its prediction, with perturbations explored via epsilon‑contamination, total variation distance, and chi‑squared divergence neighborhoods. Experiments on benchmark datasets show that the computed robustness values can serve as indicators of prediction trustworthiness and are compared against other existing indicators.
arXiv Machine Learning
Jun 30
Robust Strategic Classification under Decision-Dependent Cost Uncertainty
arXiv:2606. 30136v1 Announce Type: new Abstract: Humans facing algorithmic decision systems have been found to ``game'' them by altering their input data (at a cost to them) in order to favorably change the algorithmic outcomes they receive (at a cost to the algorithm).
arXiv Machine Learning
1d ago
Robust Non-Clairvoyant Scheduling with Classification Models
The paper tackles the single‑machine scheduling problem of minimizing total completion time in a non‑clairvoyant setting, where job processing times are unknown until completion. It introduces a robustness framework that uses a classification model’s confusion matrix to describe uncertainty as permutations within predicted classes, avoiding the computational challenges of traditional robust metrics. The authors present an optimal non‑adaptive strategy for three robust criteria and show that adaptive and randomized algorithms can outperform it when the confusion matrix has certain structural properties.
arXiv Machine Learning
Jul 8
Higher-Order Certified Robustness for Regression
arXiv:2607. 05536v1 Announce Type: cross Abstract: Randomized smoothing has emerged as a scalable technique for certifying the adversarial robustness of classifiers.
arXiv Statistics ML
Aug 25
Robust performance metrics for imbalanced classification problems
The paper demonstrates that common binary classification metrics—Matthews' correlation coefficient, Cohen's κ, the F-score, and the Jaccard similarity—are not robust to extreme class imbalance, as the Bayes classifier’s true positive rate tends to zero when the minority class proportion vanishes. To address this, the authors propose robustified versions of these metrics that include a tuning parameter, ensuring that the Bayes-optimal classifier’s threshold remains bounded and its true positive rate stays above zero even in highly imbalanced scenarios. The study provides theoretical bounds, simulation results, and practical guidance on applying these robust metrics to real data, such as a credit‑default dataset, and discusses their relationship to ROC and precision‑recall curves.