The paper revisits median‑of‑means estimation from a deterministic optimization perspective, introducing a family of block‑Lp estimators (for 0 < p ≤ 1) that achieve robust learning with heavy‑tailed and adversarially corrupted data. It shows that any convex block M‑estimator cannot attain the trimmed‑block oracle constant, while the nonconvex block‑Lp family provides finite‑sample robustness bounds that approach this oracle constant as p decreases. The authors also prove that the block‑Lp objectives have a benign landscape—every local minimum is close to the true parameter—and combine these results with block‑level concentration to obtain sub‑Gaussian deviation bounds under finite 2+δ moments, extending to high‑dimensional robust mean estimation and sparse regression.
arXiv:2201. 01973v3 Announce Type: replace-cross Abstract: The problem of linear predictions has been extensively studied for the past century under pretty generalized frameworks.
arXiv:2607. 27532v1 Announce Type: cross Abstract: Heavy tails weaken high-confidence control for the empirical mean.
arXiv:2606. 27685v1 Announce Type: cross Abstract: Pervasive data contamination -- stemming from measurement errors, outliers, or adversarial corruption -- has motivated the development of robust statistical methods.
arXiv:2607. 03871v1 Announce Type: new Abstract: Minimum maximum mean discrepancy (MMD) estimation has emerged as a robust and likelihood-free alternative to maximum likelihood estimation for parameter estimation.
The paper introduces a distributionally robust method for learning hyperparameters of first‑order convex optimization algorithms. By minimizing a Wasserstein‑robust performance estimation problem over a dataset of problem instances, the approach interpolates between classical learning‑to‑optimize (L2O) and worst‑case PEP design. The authors solve the resulting problem with stochastic gradient descent, provide high‑probability risk bounds, and demonstrate that the learned algorithms outperform both worst‑case optimal and vanilla L2O baselines on logistic regression, LASSO, and linear programming tasks.
arXiv:2605. 29497v2 Announce Type: replace Abstract: We study the problem of robustly learning Gaussian Single Index Models (SIMs) in the presence of heavy-tailed noise and a constant fraction of adversarially corrupted covariates and responses.
The paper presents a polynomial‑time algorithm for robustly learning Boolean concept classes with respect to a fixed distribution, achieving the optimal error rate of η + ε where η is the noise rate. It builds on Blanc’s earlier, computationally inefficient algorithm and introduces no‑regret learners to overcome the previous limitations. Additionally, the authors provide an efficient method that does not require an ERM oracle for any function class admitting sandwiching polynomials under hypercontractive distributions, including a first polynomial‑time solution for learning halfspaces with Gaussian marginals at error η + ε.
arXiv:2607. 10618v1 Announce Type: cross Abstract: We consider the recovery of a pair of sparse vectors from a limited number of nonlinear observations of their superposition: $y_i=g(\inner{\ba_i}{\bPhi\bw^\ast+\bPsi\bz^\ast})+e_i$, $i=1,\dots,m$, with $m\ll n$, incoherent orthonormal bases $\bPhi,\bPsi$, a scalar link $g$, and noise $e_i$ that may be heavy-tailed or contaminated.
arXiv:2505.20817v3 Announce Type: replace-cross Abstract: Gradient clipping is widely used in language-model training to control heavy-tailed gradient noise and can improve convergence guarantees ove...
arXiv:2606. 06782v1 Announce Type: cross Abstract: Robust Subspace Recovery (RSR) aims to identify an underlying d-dimensional subspace from a dataset heavily corrupted by outliers.
arXiv:2609.30499v1 Announce Type: new Abstract: Uniform noise-moment bounds exclude stochastic gradients whose variability increases with the iterate. We study ordinary, single-sample stochastic grad...
arXiv:2502. 17602v2 Announce Type: replace-cross Abstract: We study a class of stochastic nonsmooth optimization problems in which an outer variable minimizes the expectation of a pointwise maximum.