arXiv AI
Jun 9
Structured Neuron Pruning in Deep Neural Networks Using Multi-Armed Bandits
arXiv:2606. 07615v1 Announce Type: cross Abstract: Deep neural networks often contain redundant hidden units.
Structured vs. Unstructured Pruning: An Exponential Gap
arXiv:2603. 02234v3 Announce Type: replace-cross Abstract: The Strong Lottery Ticket Hypothesis (SLTH) states that large, randomly initialized neural networks contain sparse subnetworks capable of approximating a target function at initialization without training, suggesting that pruning alone is sufficient.
Related stories
Hugging Face Trending Papers
Jun 10
Finding Sparse Subnetworks in One Training Cycle via Progressive Magnitude-Based Pruning
Neural network pruning reduces model size by removing less important parameters while aiming to preserve predictive performance. Although the Lottery Ticket Hypothesis (LTH) shows that sparse subnetworks can match dense networks when trained from suitable initializations, its iterative pruning procedure requires multiple complete training cycles.
arXiv Machine Learning
Jun 11
Finding Sparse Subnetworks in One Training Cycle via Progressive Magnitude-Based Pruning
arXiv:2606. 12278v1 Announce Type: cross Abstract: Neural network pruning reduces model size by removing less important parameters while aiming to preserve predictive performance.
arXiv Machine Learning
Jun 30
HiAP: A Multi-Granular Stochastic Auto-Pruning Framework for Vision Transformers
arXiv:2603. 12222v2 Announce Type: replace-cross Abstract: Vision Transformers require significant computational resources and memory bandwidth, severely limiting their deployment on resource-constraint hardware.
arXiv Machine Learning
Jul 7
A Unified Framework for Quantized and Continuous Strong Lottery Tickets
arXiv:2607. 03860v1 Announce Type: new Abstract: The Strong Lottery Ticket Hypothesis (SLTH) asserts that sufficiently overparameterized, randomly initialized neural networks contain sparse subnetworks that, even without any training, can match the performance of a small trained network on a given dataset.
Hugging Face Trending Papers
Aug 5
Effective pruning of task-trained recurrent neural networks using noisy fluctuations and connection rescaling
The pruning of network connections is key to brain function but, despite its importance, there exist few biologically-plausible pruning rules with demonstrated good performance. In this work we evaluate noise-prune, a recently introduced unsupervised local pruning rule for recurrent networks that uses noisy fluctuations to determine the importance of connections.
arXiv Machine Learning
Aug 7
Effective pruning of task-trained recurrent neural networks using noisy fluctuations and connection rescaling
arXiv:2608. 05464v1 Announce Type: cross Abstract: The pruning of network connections is key to brain function but, despite its importance, there exist few biologically-plausible pruning rules with demonstrated good performance.
arXiv Machine Learning
Jun 11
On the Stability of Growth in Structural Plasticity
arXiv:2605. 15435v2 Announce Type: replace Abstract: Standard deep-learning pipelines usually choose the network architecture before training and keep it fixed throughout optimization.
arXiv Machine Learning
Jun 10
TENP: Trapezoidal Expert Neuron Pruning For Mixture-of-Experts
arXiv:2606. 09885v1 Announce Type: new Abstract: Mixture-of-Experts large language models (LLMs) scale efficiently through sparse activation, yet their deployment is fundamentally constrained by the large static parameter footprint of experts.
arXiv Machine Learning
Jun 25
Rational Neural Networks have Expressivity Advantages
arXiv:2602. 12390v2 Announce Type: replace Abstract: We study neural networks with trainable low-degree rational activation functions and show that they are more expressive and parameter-efficient than modern piecewise-linear and smooth activations such as ELU, LeakyReLU, LogSigmoid, PReLU, ReLU, SELU, CELU, Sigmoid, SiLU, Mish, Softplus, Tanh, Softmin, Softmax, and LogSoftmax.
arXiv Machine Learning
Jul 24
Double-Scoring: Reliable Extraction of Strong Lottery Tickets
arXiv:2607. 20555v1 Announce Type: new Abstract: The lottery ticket hypothesis proposes that large random neural networks contain sparse subnetworks that can match the performance of dense models after comparable training.
arXiv AI
Jul 1
One Shot vs. Iterative: Rethinking Pruning Strategies for Model Compression
arXiv:2508. 13836v2 Announce Type: replace-cross Abstract: Pruning is a core technique for compressing neural networks to improve computational efficiency.