arXiv Machine Learning
Aug 11
Optimal Learning Under Tsybakov Noise
arXiv:2608. 08416v1 Announce Type: new Abstract: Probably Approximately Correct (PAC) learning [Val84] is a fundamental learning model that has been extensively investigated.
Realizable Bayes-Consistency for General Metric Losses
arXiv:2605. 03823v3 Announce Type: replace Abstract: We study strong universal Bayes-consistency in the realizable setting for learning with general metric losses, extending classical characterizations beyond $0$-$1$ classification (Bousquet et al.
Related stories
arXiv Machine Learning
Jun 16
Online Realizable Regression and Applications for ReLU Networks
arXiv:2602. 19172v2 Announce Type: replace Abstract: Realizable online regression can behave very differently from online classification.
Hugging Face Trending Papers
Sep 21
Adversarially Robust PAC Learning with Optimal VC Rates
We study the problem of \emph{adversarially robust} PAC learning. In this framework, the learner observes independent samples from an unknown distribution over $\mathcal{X} \times \{0,1\}$, as in clas...
arXiv Machine Learning
Sep 22
Adversarially Robust PAC Learning with Optimal VC Rates
arXiv:2609.24260v1 Announce Type: cross Abstract: We study the problem of \emph{adversarially robust} PAC learning. In this framework, the learner observes independent samples from an unknown distrib...
arXiv Machine Learning
Sep 1
A Borel Concept Class of VC Dimension One with a Non-PAC Consistent Learner in ZFC
arXiv:2608.30246v1 Announce Type: cross Abstract: The fundamental theorem of statistical learning states that, under suitable measurability assumptions, finite Vapnik--Chervonenkis (VC) dimension gua...
arXiv Machine Learning
Jul 27
Encoding orders and trees in real-valued functions
arXiv:2607. 21761v1 Announce Type: cross Abstract: We prove function-theoretic analogues of a quantitative result of Hodges on extracting the order property from a sufficiently large 2-tree coded in a binary relation.
arXiv Machine Learning
Aug 12
Optimistic Rates for Multiclass PAC Learning
arXiv:2608. 10869v1 Announce Type: new Abstract: Worst-case multiclass bounds do not become smaller when the best classifier is already nearly correct: what is missing is an optimistic rate, a guarantee whose fluctuation scales with the oracle risk itself.
arXiv Machine Learning
Aug 28
Algorithmic Principles For Multiclass Learning Are Hard To Come By: Limits of Regularization and Proper Learning
The paper investigates fundamental limits of algorithmic principles in multiclass learning, specifically proper learning and regularization. It shows that learning cannot always be reduced to proper learning even with an enlarged hypothesis class, that proper learners may need a sublinear number of errors that can be arbitrarily large, and that regularization (SRM or local) is not universally sufficient. The authors also provide a positive theory giving sufficient conditions for SRM learnability and a characterization via integrability of revealed preferences.
arXiv Machine Learning
2d ago
Scale-Sensitive Shattering: Learnability and Evaluability at Optimal Scale
arXiv:2605.13684v2 Announce Type: replace Abstract: We study the optimal scale at which real-valued function classes exhibit uniform convergence and learnability. Our main result establishes a scale-...
arXiv Machine Learning
Sep 11
Relatively Smart II: Tractable or Semi-Supervised Instance-Optimal Learning
The paper extends the study of relatively smart learning, showing that ERM and any proper consistent learner are relatively smart for binary classification in the distribution‑free setting, achieving a quadratic sample‑complexity blowup. It further demonstrates that semi‑supervised relatively smart learning is possible with only a quadratic blowup in unlabeled data and no blowup in labeled data, though this requires a leave‑most‑out transductive approach and incurs intractability when only an agnostic ERM oracle is available. The results clarify the trade‑offs between sample efficiency, label efficiency, and computational tractability in relatively smart learning.
arXiv Machine Learning
Jul 9
The Optimal Sample Complexity of Learning Autoregressive Chain-of-Thought
arXiv:2607. 07423v1 Announce Type: new Abstract: We prove that, in the realizable PAC setting, the sample complexity of exact-trace learning for full autoregressive Chain-of-Thought traces is upper bounded by the standard multiclass rate of the local next-token class, where this rate is governed by the Daniely--Shalev-Shwartz dimension.
arXiv Machine Learning
Aug 27
Two Dimensions Govern Agnostic Multiclass Transductive Learning
arXiv:2608. 25326v1 Announce Type: new Abstract: In transductive classification, an adversary fixes a labeled population, one label is hidden uniformly, and the learner sees all remaining labels.