arXiv AI July 9, 2026 By Changcun Huang

Understanding Two-Layer Neural Networks with Smooth Activation Functions

arXiv:2507. 14177v2 Announce Type: replace-cross Abstract: This paper aims to understand the training solution, which is obtained by the back-propagation algorithm, of two-layer neural networks whose hidden layer is composed of the units with smooth activation functions, including the usual sigmoid type most commonly used before the advent of ReLUs.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI

Jul 9

On the Principles of Deep Feedforward ReLU Networks

arXiv:2607. 07035v1 Announce Type: cross Abstract: The architecture of deep feedforward neural networks is ubiquitous in deep learning, either as a whole system or as a subnetwork of other architectures, and thus its mechanism is a key ingredient of the black box of neural networks.

By Changcun Huang

arXiv Machine Learning

Jul 28

A Statistical Difference between Single-Layer Learning and Hierarchical Learning in Wide Neural Networks

arXiv:2607. 23397v1 Announce Type: new Abstract: Hierarchical neural networks are widely used in artificial intelligence, yet their mathematical properties remain incompletely understood.

By Sumio Watanabe

arXiv Machine Learning

Jun 10

Deeper or Wider: A Perspective from Optimal Generalization Error with Sobolev Loss

arXiv:2402. 00152v5 Announce Type: replace Abstract: Constructing the architecture of a neural network is a challenging pursuit for the machine learning community, and the dilemma of whether to go deeper or wider remains a persistent question.

By Yahong Yang, Juncai He

Hugging Face Trending Papers

Jul 15

Approximation of solutions of parameter-dependent problems by residual neural networks

We develop a convergent scheme to train neural networks involving analytic activation functions based on gradient flows. Convergence properties are guaranteed by Lojasiewicz theory.

arXiv Machine Learning

Jul 16

Approximation of solutions of parameter-dependent problems by residual neural networks

arXiv:2607. 13574v1 Announce Type: cross Abstract: We develop a convergent scheme to train neural networks involving analytic activation functions based on gradient flows.

By Ana Carpio

arXiv Machine Learning

Jul 14

Singular perturbations and hierarchical learning in two-layer neural networks

arXiv:2607. 10869v1 Announce Type: new Abstract: We study the population gradient flow of an infinitely wide two-layer neural network learning a misspecified single-index model in high dimension.

By C\'edric Gerbelot, Jean-Christophe Mourrat

Related stories

On the Principles of Deep Feedforward ReLU Networks

A Statistical Difference between Single-Layer Learning and Hierarchical Learning in Wide Neural Networks

Deeper or Wider: A Perspective from Optimal Generalization Error with Sobolev Loss

Approximation of solutions of parameter-dependent problems by residual neural networks

Approximation of solutions of parameter-dependent problems by residual neural networks

Singular perturbations and hierarchical learning in two-layer neural networks