arXiv Machine Learning By Chad Brown

Statistical Properties of Deep Neural Networks with Dependent Data

Read the original on arXiv Machine Learning →

The paper develops theory for deep neural network (DNN) estimators under dependent data. It establishes nonasymptotic probability bounds on the theoretical and empirical ∼2-errors of nonparametric sieve estimators for a general class of estimation problems with possibly nonstationary β-mixing data in unbounded sets. The theory is then applied to fully connected and convolutional DNN estimators without weight bounds or sparsity restrictions, deriving results for H"older smooth functions under nonstationary, subgaussian, β-mixing data with exponential or polynomial decay, and achieving the nonparametric minimax rate up to logarithmic factors in several regression settings.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI
Sep 24

Path Regularization: A Near-Complete and Optimal Nonasymptotic Generalization Theory for Multilayer Neural Networks and Double Descent Phenomenon

The paper presents a near-complete, nonasymptotic generalization theory for multilayer neural networks using path regularization, applicable to broad Lipschitz loss functions without requiring bounded loss or extreme network hyperparameters. It provides an explicit upper bound that addresses approximation rates in generalized Barron spaces and demonstrates the double descent phenomenon for ReLU networks. The authors claim near-minimax optimality for regression problems and plan to establish matching lower bounds in future work.

By Hao Yu