The paper introduces mentored decoding, a formal framework for lossy speculative decoding that can accelerate inference of autoregressive language models while potentially improving output quality. It connects this inference technique to boosting theory and extends it to all f‑divergences, revealing geometric insights for total variation, simple approximations tied to boosting compliance, and a divergence‑independent data structure enabling efficient optimal parameter queries and mentored distribution construction.
By Vivien Tran-Thien, Richard Nock
The paper investigates learning with monotone adversarial corruptions, extending previous binary classification results to multiclass and partial binary settings. It shows that even a small number of strategically inserted corrupted examples can render a learnable multiclass problem with DS dimension 2 completely unlearnable, and provides matching upper bounds when the adversary’s budget is sublinear. The work also demonstrates that classic error rates remain attainable under bounded or limited‑view adversaries.
By Julian Asilis, Shaddin Dughmi, Chirag Pabbaraju
arXiv:2606. 11149v1 Announce Type: new Abstract: We study the problem of learning a drifting concept in the presence of Massart noise.
By Mingchen Ma, Guyang Cao, Jelena Diakonikolas, Ilias Diakonikolas
arXiv:2608. 13554v1 Announce Type: new Abstract: We study online probabilistic forecasting of binary outcomes chosen by an adaptive adversary.
By Georgy Noarov, Aaron Roth
arXiv:2606. 25777v1 Announce Type: cross Abstract: We initiate a resource-aware theory of \textit{language generation in the limit} under the minimal constraint of space efficiency.
By Nicolas Flammarion, Chirag Pabbaraju, Hristo Papazov, Miltiadis Stouras, Ola Svensson
We study the problem of \emph{adversarially robust} PAC learning. In this framework, the learner observes independent samples from an unknown distribution over $\mathcal{X} \times \{0,1\}$, as in clas...
Compression is fundamental to intelligence. A model that can represent its training data as a short code has discovered regularities that enable generalization.
The paper investigates fundamental limits of algorithmic principles in multiclass learning, specifically proper learning and regularization. It shows that learning cannot always be reduced to proper learning even with an enlarged hypothesis class, that proper learners may need a sublinear number of errors that can be arbitrarily large, and that regularization (SRM or local) is not universally sufficient. The authors also provide a positive theory giving sufficient conditions for SRM learnability and a characterization via integrability of revealed preferences.
By Julian Asilis, Shaddin Dughmi, Vatsal Sharan, Alec Sun, Shang-Hua Teng, Chang Wang
arXiv:2607. 11883v1 Announce Type: new Abstract: Compression is fundamental to intelligence.
By Shikai Qiu, Marc Finzi, Yujia Zheng, Kun Zhang, Andrew Gordon Wilson
arXiv:2604. 27742v2 Announce Type: replace Abstract: A fundamental dichotomy in the theory of classification sets smoothness against statistical efficiency: smooth surrogate losses such as the logistic loss enable fast $O(1/T)$ optimization but yield slow square-root $H$-consistency bounds, while piecewise-linear losses like the Hinge loss achieve optimal linear $H$-consistency rates but are non-differentiable.
By Mehryar Mohri, Yutao Zhong
arXiv:2602. 12471v2 Announce Type: replace Abstract: We consider the optimization problem of minimizing the logistic loss with gradient descent to train a linear model for binary classification with separable data.
By Michael Crawshaw, Mingrui Liu
The paper investigates the limits of the maximal coding rate reduction (MCR²) framework for out‑of‑distribution (OOD) generalisation. It shows that MCR² can lead to complete prediction failure under distribution shift, even when a perfectly stable feature is available, and that adding invariance principles from IRM or REx does not resolve this issue. The authors conclude that additional assumptions or learning principles are needed to guarantee stable OOD predictions with MCR².
By Menghui Zhou, Gaoshan Bi, Vitaveska Lanfranchi, Po Yang