arXiv Machine Learning June 25, 2026 By Maosen Tang, Alex Townsend

Rational Neural Networks have Expressivity Advantages

Read the original on arXiv Machine Learning →

arXiv:2602. 12390v2 Announce Type: replace Abstract: We study neural networks with trainable low-degree rational activation functions and show that they are more expressive and parameter-efficient than modern piecewise-linear and smooth activations such as ELU, LeakyReLU, LogSigmoid, PReLU, ReLU, SELU, CELU, Sigmoid, SiLU, Mish, Softplus, Tanh, Softmin, Softmax, and LogSoftmax.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

llms

arXiv AI

Jul 7

Rethinking Neural Nonlinearity as Gating

arXiv:2607. 03148v1 Announce Type: cross Abstract: Activation functions are considered an essential primitive for neural nonlinearity, i.

By Muhammad Sabih, Frank Hannig, J\"urgen Teich

llms efficiency

arXiv Machine Learning

Jul 9

On Explicit Super-Expressive Approximation for Neural Networks

arXiv:2607. 06781v1 Announce Type: new Abstract: In this work, we investigate the fixed-architecture neural network approximation with explicit parameter bounds and elementary activations.

By Feng-Lei Fan, Ze-Yu Li, Chen-Yu Wang, Jian-Jun Wang

Hugging Face Trending Papers

Jul 7

On Explicit Super-Expressive Approximation for Neural Networks

In this work, we investigate the fixed-architecture neural network approximation with explicit parameter bounds and elementary activations. While prior work demonstrated super-expressive approximation using fixed-size networks, they lack quantitative and non-asymptotic characterizations of parameter magnitude with respect to the approximation error.

arXiv AI

Jul 28

Exact and Asymptotically Complete Robust Verifications of Neural Networks via Ising Solvers

arXiv:2603. 00408v2 Announce Type: replace-cross Abstract: We present an Ising-compatible framework for formal neural-network robustness verification under bounded input perturbations.

By Wenxin Li, Wenchao Liu, Weihao Li, Chuan Wang, Qi Gao, Yin Ma, Hai Wei, Kai Wen

efficiency safety

arXiv AI

Jun 12

Structured vs. Unstructured Pruning: An Exponential Gap

arXiv:2603. 02234v3 Announce Type: replace-cross Abstract: The Strong Lottery Ticket Hypothesis (SLTH) states that large, randomly initialized neural networks contain sparse subnetworks capable of approximating a target function at initialization without training, suggesting that pruning alone is sufficient.

By Davide Ferre' (CNRS, COATI, UniCA, I3S), Fr\'ed\'eric Giroire (I3S, COATI, UniCA), Frederik Mallmann-Trenn (CNRS, COATI, I3S, UniCA), Emanuele Natale (CNRS, COATI, I3S, UniCA)

efficiency safety

arXiv AI

Jul 22

Functional Equivalence and Geometric Diversity in Neural Network Approximations: An Empirical Characterization

arXiv:2607. 18930v1 Announce Type: cross Abstract: The Universal Approximation Theorem states that a neural network with a single hidden layer is sufficient to approximate any continuous univariate function on a compact domain to arbitrary error.

By Anuragine S A, Prem Jagadeesan

Related stories

Rethinking Neural Nonlinearity as Gating

On Explicit Super-Expressive Approximation for Neural Networks

On Explicit Super-Expressive Approximation for Neural Networks

Exact and Asymptotically Complete Robust Verifications of Neural Networks via Ising Solvers

Structured vs. Unstructured Pruning: An Exponential Gap

Functional Equivalence and Geometric Diversity in Neural Network Approximations: An Empirical Characterization