arXiv Machine Learning
Jul 8
Boosting with List-Decodable Codes
arXiv:2607. 05791v1 Announce Type: cross Abstract: Boosting is a fundamental technique for generically improving the accuracy of learning algorithms (Schapire 1989).
Communication Complexity of Universal Features
Related stories
arXiv AI
Jul 3
Expander Sparse Autoencoders: Parameter-Efficient Dictionaries for Mechanistic Interpretability
arXiv:2607. 01799v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) decompose internal activations of neural networks into sparse linear combinations of learned features by fitting an overcomplete dictionary $\mathbf{W}\in\mathbb{R}^{m\times n}$ with $m<n$, and inferring a sparse code $\mathbf{x}\in\mathbb{R}^n$ from $\mathbf{h}\approx\mathbf{W}\mathbf{x}$.
Hugging Face Trending Papers
Sep 21
Adversarially Robust PAC Learning with Optimal VC Rates
We study the problem of \emph{adversarially robust} PAC learning. In this framework, the learner observes independent samples from an unknown distribution over $\mathcal{X} \times \{0,1\}$, as in clas...
arXiv Machine Learning
Sep 4
A Closed-Form Formula for Consistent Lipschitz Regression on Metric Spaces with Sparse Neural Network Realizations
arXiv:2609. 03129v1 Announce Type: cross Abstract: Several classical machine-learning methods, such as KRRs and SVRs, are both computationally and analytically tractable since their estimators either admit closed-form expressions or are obtained by minimizing convex training objectives; neither feature is generally available for deep neural networks.
arXiv AI
Jun 3
Representational Capacity: Geometric Limits on Feature Representation in Transformer Language Models
arXiv:2606. 02765v1 Announce Type: cross Abstract: Model dimension ($d_{model}$) is a fundamental hyperparameter in transformer language models, yet its role in setting the geometric limits of feature representation remains under-explored.
arXiv Machine Learning
Sep 25
An Agnostic Sample Compression Scheme for Squared Loss of Near-Linear Size in the Fat-Shattering Dimension
arXiv:2609. 29696v1 Announce Type: new Abstract: We construct, for every function class $\mathcal{F}\subseteq[0,1]^{\mathcal{X}}$ and every accuracy $0<\alpha\le 1$, an agnostic sample compression scheme for the empirical squared loss: for every finite sample $S\in(\mathcal{X}\times[0,1])^m$ with arbitrary (noisy) labels, the scheme stores at most $O(\mathrm{fat}(\mathcal{F},c'\alpha)\cdot\log^3(2/\alpha))$ original labeled examples and auxiliary bits, independent of the sample size $m$, and reconstructs a function $\hat f$ with $L_2(\hat f,S)\le\inf_{f\in\mathcal{F}}L_2(f,S)+\alpha$.
arXiv Machine Learning
Aug 7
Optimal Rates for Learning with Monotone Adversaries
arXiv:2608. 06337v1 Announce Type: cross Abstract: A monotone adversary observes an i.
arXiv Machine Learning
6d ago
Approximate Solomonoff Induction: On the Robustness of Learned Universal Predictors
The paper investigates how well practical machine‑learning algorithms approximate Solomonoff’s universal predictor when trained by minimizing log‑loss over diverse datasets. It establishes tight bounds on the worst‑case increase in prediction error caused by approximating the Bayesian mixture predictor, showing that while the excess error can be large in low‑prior environments, it remains bounded on average and under distribution shifts. The authors also extend their analysis to finite‑sample settings relevant to neural networks and Transformers.
arXiv Machine Learning
Jul 3
Geometry as a Missing Axis of Representation Quality: The Variational Geometric Information Bottleneck under Data Scarcity
arXiv:2511. 02496v2 Announce Type: replace Abstract: We study latent geometry as an explicit component of representation quality in data-scarce learning.
arXiv AI
Aug 18
Optimal Lower Bounds for Networked Information Aggregation
arXiv:2608. 15472v1 Announce Type: cross Abstract: The problem of networked information aggregation, studied in Kearns et al.
arXiv Machine Learning
Jun 25
Space-Efficient Language Generation in the Limit
arXiv:2606. 25777v1 Announce Type: cross Abstract: We initiate a resource-aware theory of \textit{language generation in the limit} under the minimal constraint of space efficiency.
arXiv Machine Learning
Sep 21
On the Limits of Maximal Coding Rate Reduction for Out-of-Distribution Generalisation
The paper investigates the limits of the maximal coding rate reduction (MCR²) framework for out‑of‑distribution (OOD) generalisation. It shows that MCR² can lead to complete prediction failure under distribution shift, even when a perfectly stable feature is available, and that adding invariance principles from IRM or REx does not resolve this issue. The authors conclude that additional assumptions or learning principles are needed to guarantee stable OOD predictions with MCR².