arXiv Machine Learning

Nearly Linear Private Learning of Littlestone Classes

arXiv:2610. 23160v1 Announce Type: new Abstract: Differentially private learning aims to learn from data while protecting individual training examples.

Hugging Face Trending Papers
Aug 3

Optimal Unambiguous DNFs and Alon-Saks-Seymour

We construct unambiguous DNFs having width $O(n)$ but $0$-certificate complexity $Ω(n^2)$. By utilizing the special structure of these DNFs, we prove a lifting theorem with a constant-sized gadget that lifts the DNF to a communication problem, while losslessly translating the separation in certificate complexity to a separation in communication complexity.

arXiv Machine Learning
Sep 25

Bandit Multiclass PAC Learning: Corrected Lower Bounds, Exact Families, and a Confidence Direct-Sum Phenomenon

The paper revisits realizable multiclass PAC learning with bandit feedback, correcting a previously claimed lower bound on sample complexity. It introduces a new anchored dimension, “aBDS,” and establishes a constant‑free three‑part lower bound, while also providing tighter upper bounds that eliminate dependence on the total label count. The authors demonstrate that the optimal sample complexity can vary dramatically even among classes with identical dimensional profiles, revealing a confidence direct‑sum phenomenon and a rank‑saturation phase transition.

By Guangjian Zhang
arXiv Machine Learning
Oct 15

Dimension-Free Scaling Laws for Size Generalization: An Exponential Separation

The paper introduces a statistical learning theory for size generalization in models that handle variable-size structured data such as sets and graphs. It proves an exponential separation in sample complexity: polynomial in-domain learning yields only logarithmic decay in size-generalization error, while an exponential source condition restores polynomial decay. These results provide dimension‑free scaling laws and optimal minimax bounds for transferring learning across input sizes.

By Behrooz Tahmasebi, Melanie Weber
arXiv Machine Learning
Sep 23

Optimizing Canaries for Privacy Auditing with Metagradient Descent

The paper investigates black-box privacy auditing for differentially private learning algorithms, focusing on DP‑SGD. It introduces a method that optimizes the auditor’s canary set using metagradient descent, improving empirical lower bounds on privacy parameters compared to prior canary designs. The approach is shown to be DP‑SGD agnostic and efficient, with optimized canaries for small models remaining effective for larger DP‑SGD models.

By Matteo Boglioni, Terrance Liu, Andrew Ilyas, Zhiwei Steven Wu