Hugging Face Trending Papers
Hard Way, The Only Way: Provable Merits of Training on Hardest Examples
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Machine Learning
Oct 16
Hard Way, The Only Way: Provable Merits of Training on Hardest Examples
The paper presents a high‑dimensional theory of learning with data pruning via weighted empirical risk minimization. It analyzes pruning along a one‑dimensional subspace and derives limiting test‑error curves as input dimension and sample size grow with a fixed ratio. The authors show that, in the high‑sample regime, the optimal strategy is to keep only the 2N hardest examples, achieving a Bayes‑optimal error scaling of order N/P instead of the usual sqrt(N/P).
arXiv Computer Vision
1d ago
Is Oracle Pruning the True Oracle?
arXiv:2412.00143v3 Announce Type: replace-cross Abstract: Oracle pruning, which selects unimportant weights by minimizing the pruned train loss, has served as the foundation for most neural network p...
arXiv Machine Learning
Jun 11
RCAP: Robust, Class-Aware, Probabilistic Dynamic Dataset Pruning
arXiv:2606. 11761v1 Announce Type: new Abstract: Dynamic data pruning techniques aim to reduce computational cost while minimizing information loss by periodically selecting representative subsets of input data during model training.
arXiv Machine Learning
Jun 9
OrderDP: A Theoretically Guaranteed Lossless Dynamic Data Pruning Framework
arXiv:2606. 08574v1 Announce Type: new Abstract: Data pruning (DP), as an oft-stated strategy to alleviate heavy training burdens, reduces the volume of training samples according to a well-defined pruning method while striving for near-lossless performance.
arXiv Machine Learning
Jun 17
The Implicit Bias of Steepest Descent with Mini-batch Stochastic Gradient
arXiv:2602. 11557v2 Announce Type: replace Abstract: A variety of widely used optimization methods like SignSGD and Muon can be interpreted as instances of steepest descent under different norm-induced geometries.
arXiv Machine Learning
Jun 10
Efficiently Learning Drifting Halfspaces with Massart Noise
arXiv:2606. 11149v1 Announce Type: new Abstract: We study the problem of learning a drifting concept in the presence of Massart noise.