arXiv Machine Learning

An Exposition of GPT Astra's Proof of Lower Bound on DP Continual Counting

The note provides a detailed proof of Astra’s lower bound for differentially private continual counting, building on recent work by Harrison and Leeman. It discusses earlier results, including a Ω(√{3}√{log(n)}) bound by Bairaktari and Larsen and their subsequent Ω(log^2(n)) bound for pure differential privacy. The authors aim to offer a more natural and accessible proof, hoping to aid further research in the area.

arXiv Machine Learning
Sep 21

The Binary Tree Mechanism is Optimal for Differentially Private Continual Counting

arXiv:2607. 00876v3 Announce Type: replace-cross Abstract: Private continual counting is a fundamental problem in differential privacy: given a binary stream of length $n$, where each $1$ corresponds to the contribution of one individual, the goal is to release all running counts while protecting the privacy of each individual.

By Konstantina Bairaktari, Markus Engelund Dahl, Kasper Green Larsen
arXiv Machine Learning
Jul 2

The Binary Tree Mechanism is Optimal for Approximate Differentially Private Continual Counting

arXiv:2607. 00876v1 Announce Type: cross Abstract: Private continual counting is a fundamental problem in differential privacy: given a binary stream of length $n$, where each $1$ corresponds to the contribution of one individual, the goal is to release all running counts while protecting the privacy of each individual.

By Konstantina Bairaktari, Kasper Green Larsen
arXiv Machine Learning
Sep 17

Breaking the $T^{2/3}$ Barrier for Sequential Calibration

arXiv:2406. 13668v4 Announce Type: replace Abstract: A set of probabilistic forecasts is calibrated if each prediction of the forecaster closely approximates the empirical distribution of outcomes on the subset of timesteps where that prediction was made.

By Yuval Dagan, Constantinos Daskalakis, Maxwell Fishelson, Noah Golowich, Robert Kleinberg, Princewill Okoroafor
arXiv Machine Learning
Aug 18

Differentially Private Verification of Distribution Properties

arXiv:2604. 10819v2 Announce Type: replace-cross Abstract: A recent line of work initiated by Chiesa and Gur and further developed by Herman and Rothblum investigates the sample and communication complexity of verifying properties of distributions with the assistance of a powerful, knowledgeable, but untrusted prover.

By Elbert Du, Cynthia Dwork, Pranay Tankala, Linjun Zhang
arXiv Machine Learning
Sep 3

Improved Gradient Descent Lower Bounds Beyond Nesterov

The paper investigates the limits of accelerating gradient descent (GD) using predetermined step sizes in smooth convex optimization. It establishes new lower bounds: an ≥·n−1.6342 non‑anytime bound and an ≥·n−1.2408 anytime bound, surpassing previous results. These findings also demonstrate a strict separation between convergence exponents achievable in non‑anytime versus anytime settings.

By Yuhan Ye, Kaizhao Liu
arXiv Machine Learning
Sep 2

Dense Weak Hiding: Closing Complexity Gaps in Nonconvex and PL Finite-Sum Optimization under Individual Smoothness

The paper establishes the optimal incremental first‑order oracle (IFO) complexity for nonconvex finite‑sum optimization under individual smoothness, proving a matching lower bound that closes a previously missing √{n} factor. It also refines the analysis of the PAGE algorithm under the global Polyak‑Lojasiewicz condition, providing tighter guarantees for different ranges of the condition number. The authors introduce a novel dense weak hiding construction that yields these lower bounds and demonstrates the limits of existing methods.

By Yuxing Peng, Zhiqing Tang, Weijia Jia
arXiv Machine Learning
Sep 25

Bandit Multiclass PAC Learning: Corrected Lower Bounds, Exact Families, and a Confidence Direct-Sum Phenomenon

The paper revisits realizable multiclass PAC learning with bandit feedback, correcting a previously claimed lower bound on sample complexity. It introduces a new anchored dimension, “aBDS,” and establishes a constant‑free three‑part lower bound, while also providing tighter upper bounds that eliminate dependence on the total label count. The authors demonstrate that the optimal sample complexity can vary dramatically even among classes with identical dimensional profiles, revealing a confidence direct‑sum phenomenon and a rank‑saturation phase transition.

By Guangjian Zhang
arXiv Machine Learning
Sep 25

An Agnostic Sample Compression Scheme for Squared Loss of Near-Linear Size in the Fat-Shattering Dimension

arXiv:2609. 29696v1 Announce Type: new Abstract: We construct, for every function class $\mathcal{F}\subseteq[0,1]^{\mathcal{X}}$ and every accuracy $0<\alpha\le 1$, an agnostic sample compression scheme for the empirical squared loss: for every finite sample $S\in(\mathcal{X}\times[0,1])^m$ with arbitrary (noisy) labels, the scheme stores at most $O(\mathrm{fat}(\mathcal{F},c'\alpha)\cdot\log^3(2/\alpha))$ original labeled examples and auxiliary bits, independent of the sample size $m$, and reconstructs a function $\hat f$ with $L_2(\hat f,S)\le\inf_{f\in\mathcal{F}}L_2(f,S)+\alpha$.

By Guangjian Zhang