arXiv Machine Learning
Aug 7
Optimal Rates for Learning with Monotone Adversaries
arXiv:2608. 06337v1 Announce Type: cross Abstract: A monotone adversary observes an i.
When Does More Correct Data Hurt? Insertion-Stability and the Limits of Dimension-Based Theory
arXiv:2608. 14020v1 Announce Type: new Abstract: Adding data known to be correct ought to be safe.
Related stories
arXiv Machine Learning
Aug 12
Optimistic Rates for Multiclass PAC Learning
arXiv:2608. 10869v1 Announce Type: new Abstract: Worst-case multiclass bounds do not become smaller when the best classifier is already nearly correct: what is missing is an optimistic rate, a guarantee whose fluctuation scales with the oracle risk itself.
arXiv Machine Learning
Jul 9
The Optimal Sample Complexity of Learning Autoregressive Chain-of-Thought
arXiv:2607. 07423v1 Announce Type: new Abstract: We prove that, in the realizable PAC setting, the sample complexity of exact-trace learning for full autoregressive Chain-of-Thought traces is upper bounded by the standard multiclass rate of the local next-token class, where this rate is governed by the Daniely--Shalev-Shwartz dimension.
arXiv Machine Learning
Jul 20
ADS-C: Antidistillation Sampling for Classification
arXiv:2607. 15467v1 Announce Type: new Abstract: Knowledge distillation enables an adversary to replicate a proprietary classifier by querying its prediction interface and training a surrogate on the returned probability vectors.
arXiv Machine Learning
Jun 29
Surprises in Proper Positive-Only Learning
arXiv:2606. 28309v1 Announce Type: cross Abstract: Binary classification from positive-only samples is a variant of PAC learning in which the learner receives i.
arXiv Machine Learning
Jul 17
CASP: Learning-Augmented Offline Approximation with Verifiable Certificates and Bounded-Loss PAC Guarantees
arXiv:2607. 14545v1 Announce Type: new Abstract: Machine-learned predictions can speed up offline NP-hard optimization, but asking a predictor what to do amounts to asking it to solve the problem, and committing an unchecked prediction forfeits every worst-case guarantee.
arXiv Machine Learning
Aug 11
Optimal Learning Under Tsybakov Noise
arXiv:2608. 08416v1 Announce Type: new Abstract: Probably Approximately Correct (PAC) learning [Val84] is a fundamental learning model that has been extensively investigated.
arXiv Machine Learning
Jul 8
Boosting with List-Decodable Codes
arXiv:2607. 05791v1 Announce Type: cross Abstract: Boosting is a fundamental technique for generically improving the accuracy of learning algorithms (Schapire 1989).
arXiv AI
Jun 30
Pessimism's Paradox: Conservative Offline Training Amplifies Reward Hacking During Online Adaptation in Reasoning Models
arXiv:2606. 30627v1 Announce Type: cross Abstract: Conservative offline training is widely advocated as a safe foundation for subsequent online adaptation: if a policy stays close to well-supported behaviour, the argument goes, it is less likely to exploit imperfections in a learned reward model.
arXiv Machine Learning
Jun 17
On Randomized Algorithms in Online Strategic Classification
arXiv:2602. 06257v2 Announce Type: replace Abstract: Online strategic classification studies settings in which agents strategically modify their features to obtain favorable predictions.
arXiv Machine Learning
Jun 15
A Complexity Measure for Active Learning in Multi-group Mean Estimation
arXiv:2606. 14690v1 Announce Type: new Abstract: We study a \emph{max-risk} objective for active learning in a multi-group mean estimation $d$-armed bandits: a learner adaptively allocates a budget of $T$ samples across $d$ groups to minimize the worst-case uncertainty index $\max_{k\in[d]}\sigma_k^2/n_k$, where $\sigma_k$ is the standard deviation of the distribution of arm $d$, and $n_k$ is the number of times arm $d$ is sampled.
arXiv Machine Learning
Aug 10
Stochastic Autoregressive Learning
arXiv:2608. 07224v1 Announce Type: new Abstract: Motivated by LLMs, which generate outputs by iteratively sampling from next-token distributions, we introduce a PAC-learning model for binary stochastic autoregressive learning.