arXiv AI August 25, 2026 By Ocean Monjur, Shahriar Kabir Nahin, Anshuman Chhabra

Revisiting the Effectiveness of LLM Pruning for Test-Time Scaling

The paper revisits the impact of pruning on large language models (LLMs) during test-time scaling (TTS). While prior work found that structured pruning degrades reasoning performance, this study shows that unstructured pruning—removing only specific redundant weights—can actually improve TTS performance on reasoning benchmarks for models s1.1-7B and Qwen3-8B, sometimes surpassing the full-weight models. The authors also examine how different layer-wise sparsity allocation strategies affect these outcomes.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jul 21

When Fewer Layers Break More Chains: Layer Pruning Harms Test-Time Scaling in LLMs

arXiv:2510. 22228v2 Announce Type: replace-cross Abstract: Layer pruning has emerged as a widely adopted technique for improving the efficiency of large language models (LLMs).

By Keyu Wang, Tian Lyu, Guinan Su, Lu Yin, Marco Canini, Jonas Geiping, Shiwei Liu

llms fine-tuning efficiency benchmarks

arXiv Machine Learning

Jul 14

Prune, Update and Trim: Robust Structured Pruning for Large Language Models

arXiv:2605. 18331v2 Announce Type: replace Abstract: Large Language Models (LLMs) have experienced significant growth and development in recent years.

By Diego Coello de Portugal Mecke, Tom Hanika, Lars Schmidth-Thieme

llms efficiency benchmarks

arXiv AI

Jul 22

Beyond Single-Dimensional Compression: The Compound Sparsity Frontier of Large Language Models

arXiv:2607. 18280v1 Announce Type: cross Abstract: Large language models (LLMs) are often compressed through static parameter pruning or dynamic token-level computation, yet aggressive sparsification can trigger rapid performance degradation beyond an essential sparsity boundary.

By Chao Han, Haozhe Hu, Xiaoyu Shen

llms nlp efficiency benchmarks

arXiv AI

Aug 5

On the Limits of Layer Pruning for Generative Reasoning in Large Language Models

arXiv:2602. 01997v3 Announce Type: replace-cross Abstract: Recent work has shown that layer pruning can effectively compress large language models (LLMs) while retaining strong performance on classification benchmarks, often with little or no finetuning.

By Safal Shrestha, Anubhav Shrestha, Minwu Kim, Aadim Nepal, Keith Ross

llms fine-tuning efficiency benchmarks

arXiv Machine Learning

Jun 9

Beyond FLOPs: Benchmarking Real Inference Acceleration of LLM Pruning under a GEMM-Centric Taxonomy

arXiv:2606. 09080v1 Announce Type: new Abstract: Pruning has emerged as a dominant paradigm for accelerating large language model (LLM) inference, spanning a broad spectrum of methods that remove computation across tokens, layers, heads, dimensions, and attention patterns.

By Haozhe Hu, Hao Wu, Anhao Zhao, Longwei Ding, Peiran Yin, Yunpu Ma, Xiaoyu Shen

llms nlp efficiency benchmarks

arXiv Machine Learning

Jul 28

Compressing LLMs with MoP: Mixture of Pruners

arXiv:2602. 06127v2 Announce Type: replace Abstract: The high computational demands of Large Language Models (LLMs) motivate methods that reduce parameter count and accelerate inference.

By Bruno Lopes Yamamoto, Lucas Lauton de Alcantara, Victor Zacarias, Leandro Giusti Mugnaini, Keith Ando Ogawa, Lucas Pellicer, Rosimeire Pereira Costa, Edson Bollis, Anna Helena Reali Costa, Artur Jordao

llms fine-tuning efficiency multimodal

One email a morning, machine-written

Related stories

When Fewer Layers Break More Chains: Layer Pruning Harms Test-Time Scaling in LLMs

Prune, Update and Trim: Robust Structured Pruning for Large Language Models

Beyond Single-Dimensional Compression: The Compound Sparsity Frontier of Large Language Models

On the Limits of Layer Pruning for Generative Reasoning in Large Language Models

Beyond FLOPs: Benchmarking Real Inference Acceleration of LLM Pruning under a GEMM-Centric Taxonomy

Compressing LLMs with MoP: Mixture of Pruners