arXiv Machine Learning June 9, 2026 By Haozhe Hu, Hao Wu, Anhao Zhao, Longwei Ding, Peiran Yin, Yunpu Ma, Xiaoyu Shen

Beyond FLOPs: Benchmarking Real Inference Acceleration of LLM Pruning under a GEMM-Centric Taxonomy

Read the original on arXiv Machine Learning →

arXiv:2606. 09080v1 Announce Type: new Abstract: Pruning has emerged as a dominant paradigm for accelerating large language model (LLM) inference, spanning a broad spectrum of methods that remove computation across tokens, layers, heads, dimensions, and attention patterns.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jul 31

WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning

arXiv:2607. 28418v1 Announce Type: cross Abstract: Pruning is a promising approach for improving the efficiency of LLMs.

By Haozhe Hu, Hao Wu, Peiran Yin, Chao Han, Yunpu Ma, Xiaoyu Shen

llms nlp efficiency benchmarks

arXiv Machine Learning

Jul 27

Unified Static-Dynamic Pruning for Efficient LLM Inference

arXiv:2607. 21985v1 Announce Type: cross Abstract: The increasing deployment of large language models (LLMs) has magnified the computational and memory bottlenecks of autoregressive decoding, where low compute intensity and bandwidth-bound kernels dominate inference cost.

By Jinhyeok Kim, Yejoon Lee, Jaeyoung Do

llms efficiency

arXiv Machine Learning

Jun 10

SpenseGPT: Practical One-shot Pruning Enabling Sparse and Dense GEMMs for LLM Inference

arXiv:2606. 10445v1 Announce Type: new Abstract: Semi-structured 2:4 sparsity is widely supported by modern accelerators, providing up to a 2x theoretical speedup.

By Jaeseong Lee, Seung-won Hwang, Samyam Rajbhandari

llms efficiency

Hugging Face Trending Papers

Jun 9

SpenseGPT: Practical One-shot Pruning Enabling Sparse and Dense GEMMs for LLM Inference

Semi-structured 2:4 sparsity is widely supported by modern accelerators, providing up to a 2x theoretical speedup. However, its strict 50% sparsity constraint often causes non-negligible accuracy degradation under post-training pruning.

llms efficiency

arXiv AI

Jul 21

Sparsity-Aware Low-Rank Representation for Efficient Fine-Tuning of Large Language Models

arXiv:2601. 16991v3 Announce Type: replace-cross Abstract: Adapting large pre-trained language models to downstream tasks often entails fine-tuning millions of parameters or deploying costly dense weight updates, which hinders their use in resource-constrained environments.

By Longteng Zhang, Sen Wu, Shuai Hou, Zhengyu Qing, Zhuo Zheng, Danning Ke, Qihong Lin, Qiang Wang, Shaohuai Shi, Xiaowen Chu

llms fine-tuning efficiency benchmarks

arXiv AI

Jul 13

Accelerating GPU Inference of Large Language Models with Moderately Unstructured Sparse Weight Matrices

arXiv:2607. 08786v1 Announce Type: cross Abstract: With the growing deployment of large language models (LLMs), LLM inference cost has become a key challenge.

By Tao Lu, Haoyu Wang, Zonghui Wang, Keshen Xiang, Jiaheng Zhang, Wenzhi Chen

llms efficiency

Related stories

WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning

Unified Static-Dynamic Pruning for Efficient LLM Inference

SpenseGPT: Practical One-shot Pruning Enabling Sparse and Dense GEMMs for LLM Inference

SpenseGPT: Practical One-shot Pruning Enabling Sparse and Dense GEMMs for LLM Inference

Sparsity-Aware Low-Rank Representation for Efficient Fine-Tuning of Large Language Models

Accelerating GPU Inference of Large Language Models with Moderately Unstructured Sparse Weight Matrices