arXiv AI
Jun 2
Neural Network Compression by Approximate Differential Equivalence
arXiv:2606. 01402v1 Announce Type: cross Abstract: Neural network compression is commonly achieved by pruning parameters based on local importance scores, e.
Partial Fusion of Neural Networks: Efficient Tradeoffs Between Ensembles and Weight Aggregation
arXiv:2605. 22350v2 Announce Type: replace Abstract: Ensembles of neural networks typically outperform individual networks but incur large computational costs, whereas weight aggregation produces less costly, yet also less accurate, aggregate models.
Related stories
arXiv AI
Jun 12
Structured vs. Unstructured Pruning: An Exponential Gap
arXiv:2603. 02234v3 Announce Type: replace-cross Abstract: The Strong Lottery Ticket Hypothesis (SLTH) states that large, randomly initialized neural networks contain sparse subnetworks capable of approximating a target function at initialization without training, suggesting that pruning alone is sufficient.
arXiv AI
Aug 10
Hidden Gauge Controls Feature Specialization in ReLU Networks
arXiv:2608. 06766v1 Announce Type: cross Abstract: Training changes a network's predictions while allocating task-relevant structure across its internal units.
arXiv Machine Learning
Jul 30
Surrogate assisted diversity estimation in neural ensemble search
arXiv:2607. 26940v1 Announce Type: new Abstract: Ensembles are a standard way to improve the performance and robustness of deep neural networks, but their effectiveness crucially depends on both the quality and the diversity of individual models.
arXiv AI
Jul 22
Functional Equivalence and Geometric Diversity in Neural Network Approximations: An Empirical Characterization
arXiv:2607. 18930v1 Announce Type: cross Abstract: The Universal Approximation Theorem states that a neural network with a single hidden layer is sufficient to approximate any continuous univariate function on a compact domain to arbitrary error.
Hugging Face Trending Papers
Jul 29
Surrogate assisted diversity estimation in neural ensemble search
Ensembles are a standard way to improve the performance and robustness of deep neural networks, but their effectiveness crucially depends on both the quality and the diversity of individual models. Most neural architecture search (NAS) methods are computationally expensive.
arXiv AI
Jun 2
DOT-MoE: Differentiable Optimal Transport for MoEfication
arXiv:2606. 01666v1 Announce Type: cross Abstract: The scaling of Large Language Models (LLMs) has driven significant performance gains but created substantial challenges in inference efficiency.
Hugging Face Trending Papers
Jul 21
Functional Equivalence and Geometric Diversity in Neural Network Approximations: An Empirical Characterization
The Universal Approximation Theorem states that a neural network with a single hidden layer is sufficient to approximate any continuous univariate function on a compact domain to arbitrary error. However, the uniqueness of such neural network representations is not guaranteed, raising questions about practical identifiability.
arXiv AI
Jul 7
Causal Mechanism Reduction: Mechanism Replacement for Neural Network Pruning and Abstraction
arXiv:2602. 24266v2 Announce Type: replace-cross Abstract: Which internal mechanisms of a neural network can be replaced while preserving the computation it performs?
arXiv Machine Learning
Jun 5
Separation Power of Equivariant Neural Networks
arXiv:2406. 08966v3 Announce Type: replace Abstract: The separation power of a machine learning model refers to its ability to distinguish between different inputs and is often used as a proxy for its expressivity.
arXiv Machine Learning
Jun 10
TENP: Trapezoidal Expert Neuron Pruning For Mixture-of-Experts
arXiv:2606. 09885v1 Announce Type: new Abstract: Mixture-of-Experts large language models (LLMs) scale efficiently through sparse activation, yet their deployment is fundamentally constrained by the large static parameter footprint of experts.
arXiv Machine Learning
Aug 10
The Sparsity Whisperer
arXiv:2608. 06630v1 Announce Type: new Abstract: Pruning reduces the inference cost of large language models, but existing criteria primarily preserve large activations or reconstruct layer outputs.