arXiv AI
Data-Free Pruning of Self-Attention Layers in LLMs
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Aug 11
High-Layer Attention Pruning with Rescaling
arXiv:2507. 01900v3 Announce Type: replace-cross Abstract: Pruning is a highly effective approach for compressing large language models (LLMs), significantly reducing inference latency.
arXiv Machine Learning
Jul 31
Efficient LLMs with AMP: Attention Heads and MLP Pruning
arXiv:2504. 21174v2 Announce Type: replace Abstract: Deep learning drives a new wave in computing systems and triggers the automation of increasingly complex problems.
arXiv Machine Learning
Jul 14
Prune, Update and Trim: Robust Structured Pruning for Large Language Models
arXiv:2605. 18331v2 Announce Type: replace Abstract: Large Language Models (LLMs) have experienced significant growth and development in recent years.
arXiv Machine Learning
Jul 21
NIRVANA: Structured Pruning Reimagined for Large Language Model Compression
arXiv:2509. 14230v2 Announce Type: replace Abstract: While structured pruning presents a highly effective pathway for accelerating Large Language Model (LLM) inference, existing methods frequently suffer from significant performance degradation and demand computationally retraining to recover capabilities.
arXiv Computation and Language
Sep 22
Garbage Attention in Large Language Models: BOS Sink Heads and Sink-aware Pruning
arXiv:2601.06787v2 Announce Type: replace Abstract: Large Language Models (LLMs) are known to contain significant redundancy, yet a systematic explanation for why certain components, particularly in...
arXiv Machine Learning
Jun 25
Don't Go Breaking My LLM: The Impact of Pruning Attention Layers on Explanation Faithfulness and Confidence Calibration
arXiv:2606. 24970v1 Announce Type: new Abstract: Pruning Large Language Models (LLMs) reduces memory and inference costs by removing parts of the network, producing smaller models that retain most of their accuracy.