arXiv AI
Jul 1
One Shot vs. Iterative: Rethinking Pruning Strategies for Model Compression
arXiv:2508. 13836v2 Announce Type: replace-cross Abstract: Pruning is a core technique for compressing neural networks to improve computational efficiency.
MOONSHOT : A Framework for Multi-Objective Pruning of Vision and Large Language Models
Related stories
arXiv Computation and Language
4d ago
Pruning Laws for Large Language Models
arXiv:2504.04342v2 Announce Type: replace Abstract: Scaling up model parameters and training data consistently improves the performance of large language models (LLMs), but at the cost of rapidly gro...
arXiv Machine Learning
Aug 10
Prune Once: Retraining-Free Task-Agnostic Pruning for Vision-Language Models
arXiv:2608. 06901v1 Announce Type: cross Abstract: Vision-language models (VLMs) have achieved remarkable generalization across diverse multimodal tasks through large-scale pre-training, yet their rapidly increasing computational and memory requirements pose significant challenges for deployment in constrained environments.
arXiv Machine Learning
Jul 21
NIRVANA: Structured Pruning Reimagined for Large Language Model Compression
arXiv:2509. 14230v2 Announce Type: replace Abstract: While structured pruning presents a highly effective pathway for accelerating Large Language Model (LLM) inference, existing methods frequently suffer from significant performance degradation and demand computationally retraining to recover capabilities.
arXiv Machine Learning
Aug 10
The Sparsity Whisperer
arXiv:2608. 06630v1 Announce Type: new Abstract: Pruning reduces the inference cost of large language models, but existing criteria primarily preserve large activations or reconstruct layer outputs.
arXiv AI
Jul 21
CoCurve: Cross-Module Co-Pruning Curvature for Training-Free Structured LLM Pruning
arXiv:2607. 17568v1 Announce Type: cross Abstract: Structured pruning compresses large language models (LLMs) by removing whole computational units, such as attention heads and feed-forward (FFN) channel groups.
arXiv Computer Vision
Aug 27
SHIFT-LLM: Distribution Shift Correction in Depth-Pruned LLMs
SHIFT-LLM is a training‑free post‑pruning correction framework that inserts a Linear Residual Adapter (LRA) at each depth‑pruned site in large language models. Each LRA preserves the original residual identity while adding a lightweight affine correction calibrated via closed‑form least‑squares regression on a small held‑out set, thereby approximating the hidden state that would have been produced by the removed block. Experiments across multiple model families and benchmarks show that SHIFT‑LLM consistently recovers accuracy lost to depth pruning, achieving gains up to +15.7 points on Llama‑3.1‑8B‑Instruct with only a few hundred calibration samples and no gradient computation.
arXiv Machine Learning
Aug 11
Domain-Aware Pruning: Sparsity and Domain Generalization via Regularized Probabilistic Masking
arXiv:2608. 08624v1 Announce Type: new Abstract: Domain generalization (DG) and neural network pruning are conventionally treated as distinct objectives, targeting out-of-distribution (OOD) robustness and model efficiency, respectively.
arXiv Machine Learning
Jul 14
Prune, Update and Trim: Robust Structured Pruning for Large Language Models
arXiv:2605. 18331v2 Announce Type: replace Abstract: Large Language Models (LLMs) have experienced significant growth and development in recent years.
Hugging Face Trending Papers
Jun 22
Hybrid Compression: Integrating Pruning and Quantization for Optimized Neural Networks
Deep neural networks have witnessed remarkable advancements in recent years and have become integral to various applications. However, alongside these developments, training and deployment of neural network models on embedding and edge devices face significant challenges due to limited memory and computational resources.
arXiv Machine Learning
Jun 30
HiAP: A Multi-Granular Stochastic Auto-Pruning Framework for Vision Transformers
arXiv:2603. 12222v2 Announce Type: replace-cross Abstract: Vision Transformers require significant computational resources and memory bandwidth, severely limiting their deployment on resource-constraint hardware.
arXiv Machine Learning
Jul 31
WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning
arXiv:2607. 28418v1 Announce Type: cross Abstract: Pruning is a promising approach for improving the efficiency of LLMs.