The paper investigates the consistency of surrogate loss methods for classification and policy learning when the set of admissible classifiers is constrained, such as by interpretability or fairness requirements. It shows that hinge loss is the only surrogate that preserves consistency when constraints limit only the prediction set, but consistency can fail if constraints also restrict the functional form. The authors derive conditions guaranteeing consistency for hinge-risk-minimizing classifiers and use these results to design efficient hinge-loss-based procedures for monotone classification problems.
By Toru Kitagawa, Shosei Sakaguchi, Aleksey Tetenov
arXiv:2602. 12471v2 Announce Type: replace Abstract: We consider the optimization problem of minimizing the logistic loss with gradient descent to train a linear model for binary classification with separable data.
By Michael Crawshaw, Mingrui Liu
arXiv:2606. 19105v1 Announce Type: new Abstract: We study PAC-Bayes derandomization for smooth loss functions.
By Alexandre Lemire Paquin, Brahim Chaib-Draa, Philippe Gigu\`ere
The paper investigates minimal‑norm interpolation and λ2‑regularized logistic‑loss minimization for binary classification using univariate two‑layer ReLU networks. It provides exact geometric characterizations of optimal classifiers, showing that unpenalized hidden‑layer biases yield continuous piecewise‑affine functions that tightly follow label switches, while penalized biases produce a unique, sparsest classifier with a single kink per same‑label segment. Adding a free affine skip connection does not change these function‑space solutions but guarantees that every KKT point becomes globally optimal, eliminating suboptimal KKT points that can arise without the skip connection.
By Karolina Drabik, Ben Lewis, Antoni Puch, Etienne Boursier, Piotr Hofman, Matthias Englert, Ranko Lazi\'c
arXiv:2606. 02876v1 Announce Type: new Abstract: Randomized smoothing (RS) uses a smoothed classifier to provide architecture-agnostic certificates of $\ell_2$ classification robustness, but its dependence on per-input Monte Carlo (MC) sampling undermines its use in real-time systems.
By Jong-Ik Park, Shreyas Chaudhari, Carlee Joe-Wong, Jos\'e M. F. Moura
arXiv:2606. 19587v1 Announce Type: cross Abstract: We propose a scalable method for training prediction (machine learning) models in the predict-then-optimize paradigm, where model outputs serve as coefficients for a subsequent linear optimization task.
By Beichen Wan, Mo Liu
arXiv:2505.20817v3 Announce Type: replace-cross
Abstract: Gradient clipping is widely used in language-model training to control heavy-tailed gradient noise and can improve convergence guarantees ove...
By Taha El Bakkali El Kadi, Savelii Chezhegov, Aleksandr Beznosikov, Samuel Horv\'ath, Eduard Gorbunov
The paper investigates the difference between cost‑agnostic and cost‑sensitive loss functions when model capacity is limited. It shows that, unlike in ideal infinite‑capacity settings, optimizing a cost‑sensitive objective can yield a strictly better downstream decision than post‑processing a cost‑agnostic model. The authors prove this gap under a hypothesis class that can recover the optimal decision boundary but not the optimal cost‑agnostic hypothesis, and provide a simple example and empirical evidence on UCI datasets with simple models.
By Jessica Finocchiaro, Sanket Shah, Milind Tambe
The paper investigates binary classification with abstention under separate class‑conditional error constraints, aiming to minimize abstention while keeping both error types below specified thresholds. It derives the distribution‑free minimax rate of excess abstention risk, introduces surrogate‑loss formulations for computational feasibility with models like neural networks, and provides finite‑sample guarantees for excess surrogate ambiguity risk. The authors also formulate the learning task as a constrained optimization problem, analyze its computational complexity in the convex setting, and empirically evaluate the approach against a competing method on several datasets.
By Mohammadreza M. Kalan, Yuyang Deng, Sanaz Hamidi
We study PAC-Bayes derandomization for smooth loss functions. Our goal is to obtain generalization bounds that hold with high probability for deterministic predictors by exploiting smoothness properties of both the loss and the predictor class.
arXiv:2407. 00966v3 Announce Type: replace Abstract: In traditional models of supervised learning, the goal of a learner-- given examples from an arbitrary joint distribution on $\mathbb{R}^d \times \{\pm 1\}$-- is to output a hypothesis that is competitive (to within $\epsilon$) of the best fitting concept from some class.
By Gautam Chandrasekaran, Adam Klivans, Vasilis Kontonis, Raghu Meka, Konstantinos Stavropoulos
arXiv:2607. 05791v1 Announce Type: cross Abstract: Boosting is a fundamental technique for generically improving the accuracy of learning algorithms (Schapire 1989).
By Addison Prairie, Li-Yang Tan