arXiv:2606. 12278v1 Announce Type: cross Abstract: Neural network pruning reduces model size by removing less important parameters while aiming to preserve predictive performance.
Neural network pruning reduces model size by removing less important parameters while aiming to preserve predictive performance. Although the Lottery Ticket Hypothesis (LTH) shows that sparse subnetworks can match dense networks when trained from suitable initializations, its iterative pruning procedure requires multiple complete training cycles.
arXiv:2311.04902v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) with billions of parameters are prime targets for network pruning, which removes some model weights without hurt...
arXiv:2508. 13836v2 Announce Type: replace-cross Abstract: Pruning is a core technique for compressing neural networks to improve computational efficiency.
arXiv:2504.04342v2 Announce Type: replace Abstract: Scaling up model parameters and training data consistently improves the performance of large language models (LLMs), but at the cost of rapidly gro...
arXiv:2606. 07615v1 Announce Type: cross Abstract: Deep neural networks often contain redundant hidden units.
arXiv:2605. 15435v2 Announce Type: replace Abstract: Standard deep-learning pipelines usually choose the network architecture before training and keep it fixed throughout optimization.
arXiv:2606. 14150v1 Announce Type: new Abstract: Pruning promises a shortcut to strong small language models.
The paper presents a high‑dimensional theory of learning with data pruning via weighted empirical risk minimization. It analyzes pruning along a one‑dimensional subspace and derives limiting test‑error curves as input dimension and sample size grow with a fixed ratio. The authors show that, in the high‑sample regime, the optimal strategy is to keep only the 2N hardest examples, achieving a Bayes‑optimal error scaling of order N/P instead of the usual sqrt(N/P).
The paper introduces Contrastive Set-Difference (CSD), a pruning method that targets neurons involved in redundant reasoning by contrasting activations on long versus short correct solutions. CSD is combined with an extended preference-based chain-of-thought compression that also considers problems a pruned model always or never solves. Applied to a 7B model, the approach reduces thinking tokens by 8–22% on math benchmarks while maintaining accuracy within 0.5 points compared to traditional pruning and compression pipelines.
arXiv:2609.10311v1 Announce Type: cross Abstract: The lottery ticket hypothesis posits the existence of winning tickets: sparse subnetworks that, when trained in isolation from their original initial...
arXiv:2605. 18331v2 Announce Type: replace Abstract: Large Language Models (LLMs) have experienced significant growth and development in recent years.