arXiv Machine Learning
Unlocking Structured Pruning for Reasoning Language Models via Attention-Preserving MLP Trimming
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Aug 5
On the Limits of Layer Pruning for Generative Reasoning in Large Language Models
arXiv:2602. 01997v3 Announce Type: replace-cross Abstract: Recent work has shown that layer pruning can effectively compress large language models (LLMs) while retaining strong performance on classification benchmarks, often with little or no finetuning.
arXiv AI
Aug 25
Revisiting the Effectiveness of LLM Pruning for Test-Time Scaling
The paper revisits the impact of pruning on large language models (LLMs) during test-time scaling (TTS). While prior work found that structured pruning degrades reasoning performance, this study shows that unstructured pruning—removing only specific redundant weights—can actually improve TTS performance on reasoning benchmarks for models s1.1-7B and Qwen3-8B, sometimes surpassing the full-weight models. The authors also examine how different layer-wise sparsity allocation strategies affect these outcomes.
arXiv Machine Learning
Aug 11
High-Layer Attention Pruning with Rescaling
arXiv:2507. 01900v3 Announce Type: replace-cross Abstract: Pruning is a highly effective approach for compressing large language models (LLMs), significantly reducing inference latency.
arXiv Computation and Language
2d ago
To Reason but Not to Stop: Understanding and Recovering Pruned Large Reasoning Models
arXiv:2610.19934v1 Announce Type: new Abstract: Weight pruning can degrade the accuracy of large reasoning models (LRMs), yet it remains unclear whether it damages reasoning itself. We find that much...
arXiv Machine Learning
Jul 14
Prune, Update and Trim: Robust Structured Pruning for Large Language Models
arXiv:2605. 18331v2 Announce Type: replace Abstract: Large Language Models (LLMs) have experienced significant growth and development in recent years.
arXiv AI
Jul 14
OS-Pruner: Pruning Chains-of-Thought of Reasoning Models via Optimal Stopping
arXiv:2607. 11089v1 Announce Type: new Abstract: Large Language Models (LLMs) have achieved remarkable success in complex reasoning tasks through Chain-of-Thought (CoT) prompting.