The paper investigates how the magnitude of numerical errors in a learning system should be evaluated, arguing that error impact depends on the current learning state, the class affected, and the class weight. By analyzing softmax cross‑entropy, the authors derive exact bounds on loss changes for fixed error sets and use these to provide finite‑error guarantees for losses, probabilities, predictions, and gradients. They then invert these guarantees to certify the maximum allowable primitive error for a given learning‑level requirement, demonstrating through von Mises‑Fisher learning that identical errors can have vastly different effects across states.
By Ningkang Peng, Qianfeng Yu, Jingyang Mao, Xiaoqian Peng, Yanhui Gu
The paper demonstrates that in differentiable learning, the scalar objective computed during the forward pass can differ from the gradient supplied in the backward pass when probabilistic objectives depend on finite special‑function recurrences, custom backward rules, and numerical clipping. This mismatch is shown in high‑dimensional von Mises‑Fisher learning, where identical forward scores can lead to different gradients and optimization paths. To address this, the authors introduce AR/FR, a fixed‑depth analytic method that jointly constructs a potential and its derivative, guaranteeing forward‑backward coherence and providing a uniform cubic‑order error bound relative to the exact Bessel ratio across all nonnegative concentration values.
By Ningkang Peng, Xiaoqian Peng, Yifan He, Anjie Hu, Chao Tan, Peirong Ma, Yanhui Gu
The paper investigates the difference between cost‑agnostic and cost‑sensitive loss functions when model capacity is limited. It shows that, unlike in ideal infinite‑capacity settings, optimizing a cost‑sensitive objective can yield a strictly better downstream decision than post‑processing a cost‑agnostic model. The authors prove this gap under a hypothesis class that can recover the optimal decision boundary but not the optimal cost‑agnostic hypothesis, and provide a simple example and empirical evidence on UCI datasets with simple models.
By Jessica Finocchiaro, Sanket Shah, Milind Tambe
arXiv:2609.33291v2 Announce Type: replace-cross
Abstract: Many generative systems return multiple candidate responses and are evaluated according to the best one. Recent work shows that, when demonst...
By Chandramauli Chakraborty, Cong Ma
arXiv:2602. 21160v3 Announce Type: replace-cross Abstract: In safety-critical classification, the cost of failure is often asymmetric, yet Bayesian deep learning summarises epistemic uncertainty with a single scalar, mutual information (MI), that cannot distinguish whether a model's ignorance involves a benign or safety-critical class.
By Mame Diarra Toure, David A. Stephens
arXiv:2605.22940v2 Announce Type: replace-cross
Abstract: Entropy reduction is not a safety criterion: it can leave predictions unchanged or accompany increasing risk for a protected group. We study...
By Kim Phuc Tran
arXiv:2511. 22823v2 Announce Type: replace-cross Abstract: Weakly supervised learning has emerged as a practical alternative to fully supervised learning when complete and accurate labels are costly or infeasible to acquire.
By Miao Zhang, Junpeng Li, Changchun Hua, Yana Yang
arXiv:2607.22258v2 Announce Type: replace
Abstract: Models trained on long-tailed data using standard softmax tend to exhibit higher training error and a larger generalisation gap for classes with fe...
By Yi-Hang Zhu, Rajeev Raman, Shiqi Su, Jianyuan Sun, Xinyu Yang, Nan Xing, Huiyu Zhou
The paper identifies two numerical bugs in standard softmax, log‑softmax, and cross‑entropy implementations: Softmax Collapse, where a dominant logit causes its gradient to be rounded to zero, and Softmax Cancellation, where student and teacher logits that are equal still produce non‑zero gradients. The authors provide straightforward implementations that correct both issues. Their fix removes the previously observed "Slingshot" phenomenon, accelerates grokking with weight decay, prevents accuracy collapse without weight decay, keeps student networks aligned with teachers during distillation, and demonstrates that post‑memorization logit growth is bounded by an equilibrium between weight decay and Adam’s epsilon hyperparameter.
By Jonathan T. Barron, Dan Gnanapragasam, Michael Tschannen
arXiv:2609. 07997v1 Announce Type: new Abstract: We characterize the sharp structure-agnostic minimax risk for coefficient estimation in the partial linear model when the outcome and treatment nuisances are learned by two distinct black-box learners, which resolves the open problem in double machine learning posed by Gu (2025).
By Haichen Hu, David Simchi-Levi
arXiv:2609.38784v1 Announce Type: new
Abstract: In probabilistic contrastive learning, a shared temperature is commonly interpreted as a shared similarity scale, but this interpretation does not hold...
By Ningkang Peng, Qianfeng Yu, Jingyang Mao, Xiaoqian Peng, Tingyu Lu, Peirong Ma, Yanhui Gu
arXiv:2608. 10372v1 Announce Type: new Abstract: Post-hoc calibration aligns a classifier's predicted confidences with its empirical accuracy without retraining.
By Lening Zhao, Qipeng Zhan, Li Shen