arXiv Machine Learning

Adversarially Robust PAC Learning with Optimal VC Rates

arXiv Machine Learning
Sep 25

Bandit Multiclass PAC Learning: Corrected Lower Bounds, Exact Families, and a Confidence Direct-Sum Phenomenon

The paper revisits realizable multiclass PAC learning with bandit feedback, correcting a previously claimed lower bound on sample complexity. It introduces a new anchored dimension, “aBDS,” and establishes a constant‑free three‑part lower bound, while also providing tighter upper bounds that eliminate dependence on the total label count. The authors demonstrate that the optimal sample complexity can vary dramatically even among classes with identical dimensional profiles, revealing a confidence direct‑sum phenomenon and a rank‑saturation phase transition.

By Guangjian Zhang
arXiv Machine Learning
Nov 2

New Lower Bounds for Private PAC Learning

The paper investigates realizable PAC learning under approximate differential privacy and develops a framework for establishing lower bounds on sample complexity. It demonstrates that classes sharing the same VC and Littlestone dimensions can have different private sample complexities, introducing the Independent Class Packing (ICP) parameter to capture this distinction. Using ICP, the authors derive tight lower bounds for several hypothesis classes, including axis-aligned rectangles, unions of intervals, linear subspaces, and improved bounds for halfspaces and convex polygons, and show that ICP characterizes private PAC sample complexity for product classes up to logarithmic factors.

By Prathamesh Dharangutte, Amartya Sanyal
arXiv Machine Learning
6d ago

Learning from First Discrepancies with Joint Reset and Pilot Guarantees

The paper studies learning from runs that stop at their first incorrect decision, revealing only the queries and correct labels up to that point. It analyzes how many such runs are required to learn a hypothesis from a fixed binary class of VC dimension at most $d$, providing bounds on the number of runs and distinct pilots needed to achieve a risk estimate within ε of the optimal risk. The results include upper bounds on sample complexity, treatment of unknown cost ceilings, and matching lower bounds for finite classes.

By Michal \v{C}ert\'ik
arXiv Machine Learning
Oct 22

Approximate Solomonoff Induction: On the Robustness of Learned Universal Predictors

The paper investigates how well practical machine‑learning algorithms approximate Solomonoff’s universal predictor when trained by minimizing log‑loss over diverse datasets. It establishes tight bounds on the worst‑case increase in prediction error caused by approximating the Bayesian mixture predictor, showing that while the excess error can be large in low‑prior environments, it remains bounded on average and under distribution shifts. The authors also extend their analysis to finite‑sample settings relevant to neural networks and Transformers.

By Qian Li, Shang-Hua Teng
Hugging Face Trending Papers
Aug 13

Bagging Robustly Learns VC Classes with Linear Sample Complexity

We revisit the problem of learning predictors robust to adversarial examples at test-time. We prove that VC classes are adversarially robustly learnable with sample complexity linear in the VC dimension $d$, providing an exponential improvement over the previous upper bound of Montasser, Hanneke, and Srebro (2019).