arXiv Machine Learning By Sumit Vashishtha

Dimension-Free PAC-Bayes Generalization for Deep Neural Networks via Dirichlet Processes

Read the original on arXiv Machine Learning →

The paper introduces a dimension‑free PAC‑Bayes generalization bound for deep neural networks by placing the prior over the non‑parametric space of data distributions using a Dirichlet Process. This shift eliminates the dependence on the number of network parameters, yielding a bound that remains non‑vacuous even for over‑parameterized models. Experiments on MNIST with up to 4.2 million parameters show the new bound staying stable at 8.8 %–9.4 %, whereas classical weight‑space bounds become vacuous.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning
Sep 21

Sparse Priors for Efficient Distribution Learning

arXiv:2609. 20883v1 Announce Type: new Abstract: Despite the widespread use and success of generative AI techniques today, theoretical guarantees on learning a distribution supported in $d$ dimensions from $n$ samples degrade as $O(n^{-1/\Theta(d)})$, though shown to be minimax optimal.

By Saumya Goyal, Barnab\'as P\'oczos
arXiv Machine Learning
Sep 23

Practical Scaling Laws: Converting Compute into Performance in a Data-Constrained World

The paper introduces a new closed‑form scaling law that extends Chinchilla’s original formula to handle data‑constrained regimes. It decomposes loss into undercapacity, undertraining, and overfitting components, saturating between an irreducible loss and an uninformed baseline. The authors validate the model on diverse architectures and domains, achieving state‑of‑the‑art RMSE across multiple LLM scaling‑law grids and enabling cost‑aware training allocations.

By Christopher M. Bryant, Hao Liu