arXiv Machine Learning By Adam R. Klivans, Konstantinos Stavropoulos, Sergei Tikhonov, Arsen Vasilyan

Efficient Robust Learning at the Information-Theoretic Limit

Read the original on arXiv Machine Learning →

The paper presents a polynomial‑time algorithm for robustly learning Boolean concept classes with respect to a fixed distribution, achieving the optimal error rate of η + ε where η is the noise rate. It builds on Blanc’s earlier, computationally inefficient algorithm and introduces no‑regret learners to overcome the previous limitations. Additionally, the authors provide an efficient method that does not require an ERM oracle for any function class admitting sandwiching polynomials under hypercontractive distributions, including a first polynomial‑time solution for learning halfspaces with Gaussian marginals at error η + ε.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning
Sep 3

Smoothed Analysis for Learning Concepts with Low Intrinsic Dimension

arXiv:2407. 00966v3 Announce Type: replace Abstract: In traditional models of supervised learning, the goal of a learner-- given examples from an arbitrary joint distribution on $\mathbb{R}^d \times \{\pm 1\}$-- is to output a hypothesis that is competitive (to within $\epsilon$) of the best fitting concept from some class.

By Gautam Chandrasekaran, Adam Klivans, Vasilis Kontonis, Raghu Meka, Konstantinos Stavropoulos
arXiv Machine Learning
Nov 2

Robust Learning of Multi-Index Models under Logconcave Distributions via Weak Partial Learners

The paper introduces a new algorithmic framework for learning Boolean concept classes under logconcave distributions, focusing on multi‑index models. It leverages a boosting approach based on weak partial learners, overcoming the need for distribution‑free weak learners by using a recursive moment‑based partitioning procedure that aligns low‑degree moments of conditioned distributions with the original logconcave distribution. This yields faster robust learners for fundamental concept classes such as functions of halfspaces and multiclass linear classifiers, and includes an efficient method for detecting moment discrepancies that may be useful beyond this work.

By Ilias Diakonikolas, Daniel M. Kane, Mingchen Ma
arXiv Machine Learning
Sep 11

Relatively Smart II: Tractable or Semi-Supervised Instance-Optimal Learning

The paper extends the study of relatively smart learning, showing that ERM and any proper consistent learner are relatively smart for binary classification in the distribution‑free setting, achieving a quadratic sample‑complexity blowup. It further demonstrates that semi‑supervised relatively smart learning is possible with only a quadratic blowup in unlabeled data and no blowup in labeled data, though this requires a leave‑most‑out transductive approach and incurs intractability when only an agnostic ERM oracle is available. The results clarify the trade‑offs between sample efficiency, label efficiency, and computational tractability in relatively smart learning.

By Shaddin Dughmi, Alireza F. Pour
arXiv Machine Learning
Sep 30

Learning Distributionally Robust First-Order Methods for Convex Optimization

The paper introduces a distributionally robust method for learning hyperparameters of first‑order convex optimization algorithms. By minimizing a Wasserstein‑robust performance estimation problem over a dataset of problem instances, the approach interpolates between classical learning‑to‑optimize (L2O) and worst‑case PEP design. The authors solve the resulting problem with stochastic gradient descent, provide high‑probability risk bounds, and demonstrate that the learned algorithms outperform both worst‑case optimal and vanilla L2O baselines on logistic regression, LASSO, and linear programming tasks.

By Vinit Ranjan, Jisun Park, Bartolomeo Stellato