arXiv Machine Learning
Jul 21
Robust Losses from Univariate Base Functions for Noisy-Label Learning
arXiv:2607. 16768v1 Announce Type: new Abstract: Learning with noisy labels is a fundamental problem in training reliable deep neural networks.
Symmetrization of Loss Functions for Robust Training of Neural Networks in the Presence of Noisy Labels
arXiv:2605. 20347v2 Announce Type: replace Abstract: Labeling a training set is often expensive and susceptible to errors, making the design of robust loss functions for label noise an important problem.
Related stories
arXiv Machine Learning
Aug 11
No Unique Minimizer, No Problem: On the Consistency of Robust Neural Classifiers
arXiv:2608. 08489v1 Announce Type: new Abstract: Neural network classifiers trained by cross-entropy minimization are highly sensitive to label noise and adversarial contamination.
arXiv Machine Learning
Aug 11
Efficient and Noise-Tolerant PAC Learning of Multiclass Linear Classifiers
arXiv:2605. 18662v2 Announce Type: replace Abstract: Noise-tolerant PAC learning of linear models has been of central interests in machine learning community since the last century.
arXiv Machine Learning
Jul 23
Label-Noise Resistant Learning via Optimal Brain Damage Masking
arXiv:2508. 09697v4 Announce Type: replace Abstract: Noisy labels are inevitable in real-world multimedia applications.
arXiv Machine Learning
Sep 24
Minimal-Norm Univariate Two-Layer ReLU Classification: Exact Solutions and Global Optimality with Skip Connections
The paper investigates minimal‑norm interpolation and λ2‑regularized logistic‑loss minimization for binary classification using univariate two‑layer ReLU networks. It provides exact geometric characterizations of optimal classifiers, showing that unpenalized hidden‑layer biases yield continuous piecewise‑affine functions that tightly follow label switches, while penalized biases produce a unique, sparsest classifier with a single kink per same‑label segment. Adding a free affine skip connection does not change these function‑space solutions but guarantees that every KKT point becomes globally optimal, eliminating suboptimal KKT points that can arise without the skip connection.
arXiv Computer Vision
Sep 14
Learning Sign Language Recognition under Label Noise: A Study of Noise-Robust Losses for Isolated and Continuous Settings
The paper investigates the impact of label noise on sign language recognition, comparing robust loss functions—symmetric cross entropy (SCE) and generalized cross entropy (GCE)—to standard cross entropy (CE) in both isolated (ISLR) and continuous (CSLR) settings. Experiments on ASL Citizen with injected symmetric noise show that SCE and GCE outperform CE across multiple backbones, though GCE’s optimal hyperparameter does not transfer well. In CSLR experiments on PHOENIX-2014, robust losses offer limited gains, with performance largely influenced by auxiliary weight settings rather than the loss choice.
arXiv AI
Jun 18
Generalized Kullback-Leibler Divergence Loss
arXiv:2503. 08038v2 Announce Type: replace-cross Abstract: In this paper, we delve deeper into the Kullback-Leibler (KL) Divergence loss and mathematically prove that it is equivalent to the Decoupled Kullback-Leibler (DKL) Divergence loss that consists of (1) a weighted Mean Square Error (wMSE) loss and (2) a Cross-Entropy loss incorporating soft labels.
arXiv Machine Learning
Aug 13
Linear-Core Surrogates: Smooth Loss Functions with Linear Rates for Classification and Structured Prediction
arXiv:2604. 27742v2 Announce Type: replace Abstract: A fundamental dichotomy in the theory of classification sets smoothness against statistical efficiency: smooth surrogate losses such as the logistic loss enable fast $O(1/T)$ optimization but yield slow square-root $H$-consistency bounds, while piecewise-linear losses like the Hinge loss achieve optimal linear $H$-consistency rates but are non-differentiable.
arXiv Machine Learning
Jun 5
Towards Label-Noise Resistant Learning via Optimal Brain Damage Masking
arXiv:2508. 09697v3 Announce Type: replace Abstract: Noisy labels are inevitable in real-world scenarios.
arXiv Machine Learning
Aug 17
Convex losses and their applications to SVM, SVR, and Shallow Neural Networks
arXiv:2608. 14288v1 Announce Type: new Abstract: We propose multiple new convex losses for SVM and Neural Networks, applied to binary classification tasks.
arXiv Machine Learning
Jun 2
Exploiting weight-space symmetries for approximating curvature
arXiv:2606. 00442v1 Announce Type: new Abstract: Many machine learning techniques rely on approximating a loss function's curvature, but this is notoriously hard to do at the scale of modern deep networks.
arXiv Machine Learning
Jun 2
Robust Learning of a Group DRO Neuron
arXiv:2601. 18115v2 Announce Type: replace Abstract: We study the problem of learning a single neuron under standard squared loss in the presence of arbitrary label noise and group-level distributional shifts, for a broad family of covariate distributions.