arXiv Computation and Language
Pruning as Regularization: Sensitivity-Aware One-Shot Pruning in ASR
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
Hugging Face Trending Papers
Sep 23
Six Layers Less: Encoder Pruning for Whisper with Label-Free Recovery
Pruning large pre-trained transformer-based ASR models such as OpenAI's Whisper has seen great adoption, as pruning the decoder led to significant end-to-end transcription speedups. For instance, the...
arXiv AI
1d ago
Data-Free Pruning of Self-Attention Layers in LLMs
arXiv:2512.20636v2 Announce Type: replace-cross Abstract: Many self-attention sublayers in large language models (LLMs) can be removed with little to no loss. We attribute this to the Attention Suppr...
arXiv Machine Learning
Sep 24
Six Layers Less: Encoder Pruning for Whisper with Label-Free Recovery
The paper introduces a method to prune six layers from the encoder of OpenAI’s Whisper ASR model, reducing the encoder stack by 18.5% without requiring custom inference code. Layers are selected based on their minimal impact on Word Error Rate when removed. After pruning, the model’s WER rises from 18.2% to 21.9%, but distillation with unlabeled monolingual speech data lowers it to 20.1%. "whyItMatters":"The approach offers a straightforward way to accelerate Whisper inference by simplifying the encoder while maintaining acceptable accuracy, and the released code and model enable immediate adoption by the community."
arXiv Machine Learning
Jul 21
NIRVANA: Structured Pruning Reimagined for Large Language Model Compression
arXiv:2509. 14230v2 Announce Type: replace Abstract: While structured pruning presents a highly effective pathway for accelerating Large Language Model (LLM) inference, existing methods frequently suffer from significant performance degradation and demand computationally retraining to recover capabilities.
arXiv Machine Learning
Aug 10
The Sparsity Whisperer
arXiv:2608. 06630v1 Announce Type: new Abstract: Pruning reduces the inference cost of large language models, but existing criteria primarily preserve large activations or reconstruct layer outputs.
arXiv Machine Learning
Aug 10
Prune Once: Retraining-Free Task-Agnostic Pruning for Vision-Language Models
arXiv:2608. 06901v1 Announce Type: cross Abstract: Vision-language models (VLMs) have achieved remarkable generalization across diverse multimodal tasks through large-scale pre-training, yet their rapidly increasing computational and memory requirements pose significant challenges for deployment in constrained environments.