arXiv Machine Learning
Jun 29
Surprises in Proper Positive-Only Learning
arXiv:2606. 28309v1 Announce Type: cross Abstract: Binary classification from positive-only samples is a variant of PAC learning in which the learner receives i.
A Borel Concept Class of VC Dimension One with a Non-PAC Consistent Learner in ZFC
Related stories
arXiv Machine Learning
Jul 2
Recursively Enumerably Representable Classes and Computable Versions of the Fundamental Theorem of Statistical Learning
arXiv:2511. 02644v2 Announce Type: replace Abstract: We study computable probably approximately correct (CPAC) learning, where learners are required to be computable functions.
arXiv Machine Learning
Aug 28
Algorithmic Principles For Multiclass Learning Are Hard To Come By: Limits of Regularization and Proper Learning
The paper investigates fundamental limits of algorithmic principles in multiclass learning, specifically proper learning and regularization. It shows that learning cannot always be reduced to proper learning even with an enlarged hypothesis class, that proper learners may need a sublinear number of errors that can be arbitrarily large, and that regularization (SRM or local) is not universally sufficient. The authors also provide a positive theory giving sufficient conditions for SRM learnability and a characterization via integrability of revealed preferences.
arXiv Machine Learning
Aug 7
Realizable Bayes-Consistency for General Metric Losses
arXiv:2605. 03823v3 Announce Type: replace Abstract: We study strong universal Bayes-consistency in the realizable setting for learning with general metric losses, extending classical characterizations beyond $0$-$1$ classification (Bousquet et al.
arXiv AI
Jun 26
Unbiased Canonical Set-Valued Oracles Via Lattice Theory
arXiv:2606. 26418v1 Announce Type: new Abstract: A non-agentic "oracle" AI that estimates probabilities of future events faces a self-reference problem: once its answer is learned and acted upon, it can change the very probability it was asked to report.
arXiv Machine Learning
Jul 9
The Optimal Sample Complexity of Learning Autoregressive Chain-of-Thought
arXiv:2607. 07423v1 Announce Type: new Abstract: We prove that, in the realizable PAC setting, the sample complexity of exact-trace learning for full autoregressive Chain-of-Thought traces is upper bounded by the standard multiclass rate of the local next-token class, where this rate is governed by the Daniely--Shalev-Shwartz dimension.
arXiv Machine Learning
Aug 12
Optimistic Rates for Multiclass PAC Learning
arXiv:2608. 10869v1 Announce Type: new Abstract: Worst-case multiclass bounds do not become smaller when the best classifier is already nearly correct: what is missing is an optimistic rate, a guarantee whose fluctuation scales with the oracle risk itself.
arXiv Machine Learning
Jun 2
How Many Domains Suffice for Domain Generalization? A Tight Characterization via the Domain Shattering Dimension
arXiv:2506. 16704v3 Announce Type: replace Abstract: We study a fundamental question of domain generalization: given a family of domains (i.
arXiv AI
Aug 7
An Optimal Agnostic PAC Algorithm
arXiv:2608. 06363v1 Announce Type: cross Abstract: Let $H\subseteq\{-1,+1\}^X$ be a class of finite VC dimension $d\ge1$.
arXiv Machine Learning
Aug 27
Two Dimensions Govern Agnostic Multiclass Transductive Learning
arXiv:2608. 25326v1 Announce Type: new Abstract: In transductive classification, an adversary fixes a labeled population, one label is hidden uniformly, and the learner sees all remaining labels.
arXiv Machine Learning
Jun 8
Universal consistency of the $k$-NN rule in metric spaces and Nagata dimension. III
arXiv:2512. 17058v3 Announce Type: replace Abstract: We establish the last missing link allowing to describe those complete separable metric spaces $X$ in which the $k$ nearest neighbour classifier is universally consistent, both in combinatorial terms of dimension theory and via a fundamental property of real analysis.
arXiv Machine Learning
Aug 11
Constrained Learning with Universally Learnable Concept Classes
arXiv:2608. 08414v1 Announce Type: new Abstract: We study constrained statistical learning over infinite-dimensional hypothesis classes in the fully nonconvex setting, and establish universal PACC learnability of the solutions of dual algorithms: Probably Approximately Correct on Constraints, guaranteeing optimality and constraint satisfaction at once.