arXiv AI
OPTIMA: Optimal One-shot Pruning for LLMs via Quadratic Programming Reconstruction
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Sep 1
MOONSHOT : A Framework for Multi-Objective Pruning of Vision and Large Language Models
arXiv:2604.13287v2 Announce Type: replace Abstract: Weight pruning is a common technique for compressing large neural networks. We focus on the challenging post-training one-shot setting, where a pre...
arXiv Machine Learning
Jun 10
SpenseGPT: Practical One-shot Pruning Enabling Sparse and Dense GEMMs for LLM Inference
arXiv:2606. 10445v1 Announce Type: new Abstract: Semi-structured 2:4 sparsity is widely supported by modern accelerators, providing up to a 2x theoretical speedup.
Hugging Face Trending Papers
Jun 9
SpenseGPT: Practical One-shot Pruning Enabling Sparse and Dense GEMMs for LLM Inference
Semi-structured 2:4 sparsity is widely supported by modern accelerators, providing up to a 2x theoretical speedup. However, its strict 50% sparsity constraint often causes non-negligible accuracy degradation under post-training pruning.
arXiv Machine Learning
Oct 15
Beyond Size: How Gradients Shape Pruning Decisions in Large Language Models
arXiv:2311.04902v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) with billions of parameters are prime targets for network pruning, which removes some model weights without hurt...
arXiv AI
5d ago
Prune Gradually, Adapt Sparsely: Efficient Pruning of Large Language Models at Extreme Sparsity
arXiv:2610.41968v1 Announce Type: new Abstract: Weight pruning can reduce the storage and inference cost of large language models (LLMs), but preserving quality at high sparsity levels remains challe...
arXiv Machine Learning
Sep 30
Probe-Space Preconditioning for Fast and Stable Zero-Order Training
arXiv:2609.38095v1 Announce Type: new Abstract: Backpropagation (BP) dominates deep learning but imposes a massive memory tax. For example, training OPT-30B with Adam requires $\approx$ 600GB of GPU...