COMPOT is a training‑free compression framework for transformer models that uses a small calibration dataset to estimate a sparse weight factorization. It employs orthogonal dictionaries, enabling closed‑form Procrustes updates and analytical single‑step sparse coding, which dramatically reduces optimization cost. The method also introduces a one‑shot dynamic allocation strategy to adaptively redistribute compression rates across layers, achieving a superior quality‑compression trade‑off compared to strong low‑rank and sparse baselines while remaining compatible with post‑training quantization.
By Denis Makhov, Dmitriy Shopkhoev, Magauiya Zhussip, Ammar Ali, Baher Mohammad, Stamatios Lefkimmiatis
Prohibitive computational and environmental costs impede the scalable deployment of Large Language Models (LLMs). Traditional compression techniques (sparsity, quantization, low-rank approximations) a...
The paper introduces the "Compression Trinity," a unified framework that jointly applies sparsity, quantization, and low‑rank approximations to compress large language models. It presents several methods—MKOR, SLoPe, OPTIMA, PATCH, and SLiM—that leverage these three pillars to accelerate training, reduce memory bandwidth, and recover accuracy, achieving significant speedups and accuracy gains over existing techniques. The results demonstrate that combining all three compression strategies is essential for efficient, scalable, high‑performance LLM deployment.
By Mohammad Mozaffari
The paper introduces Trainable Vector Quantization (TVQ), a method that uses straight-through estimation to allow gradient descent optimization of vector-quantized large language models. TVQ reformulates VQ with trainable latent weights, enabling arbitrary objective minimization and is applied in a block-wise knowledge distillation framework to balance quality and training efficiency. Experiments show TVQ achieves state‑of‑the‑art low‑bit post‑training quantization, achieving near‑lossless 2‑bit quantization for LLaMA2‑70B with only a 1% accuracy drop on question‑answering benchmarks.
By Anastasiia Prutianova, Alina Bogdanova, Mikhail Shutov, Dmitrii Poimanov, Anatoly Gorkov, Ilia Zharikov
SparseDecoding is a decoding‑aware pruning framework that improves large language model inference by aligning pruning with the activations seen during actual autoregressive generation. It constructs calibration matrices from dense‑model activations (excluding prefill) and introduces an optimized N:M sparse matrix‑vector kernel with bitmask indexing to accelerate decoding. Experiments on models such as Llama‑3.1‑8B, Llama‑3.3‑70B, and Qwen3‑14B/32B show consistent performance gains over standard fixed‑text calibration and up to 1.48× wall‑clock speedup on A100 GPUs.
By Qitong Wang, Xinwei Niu, Mingluo Su, Shanwei Zhao, Shiai Zhu, Huan Wang
arXiv:2606. 07819v1 Announce Type: new Abstract: Recently, the efficiency of Large Language Models (LLMs) deployment has become a critical concern in practical applications.
By Hoang-Loc La, Truong-Thanh Le, Amir Taherkordi, Phuong Hoai Ha
The paper introduces 3BASiL-TM, a one‑shot post‑training method for compressing large language models using a sparse plus low‑rank decomposition. It employs a novel 3‑block ADMM algorithm to minimize layer‑wise reconstruction error and a transformer‑matching refinement that jointly optimizes sparse and low‑rank components across layers. Experiments show a >30% reduction in WikiText2 perplexity gap for LLaMA‑8B under a (2:4 sparse + 64 LR) setting and a 2.5× faster compression runtime on an A100 GPU compared to state‑of‑the‑art methods.
By Mehdi Makni, Xiang Meng, Rahul Mazumder
arXiv:2601. 16991v3 Announce Type: replace-cross Abstract: Adapting large pre-trained language models to downstream tasks often entails fine-tuning millions of parameters or deploying costly dense weight updates, which hinders their use in resource-constrained environments.
By Longteng Zhang, Sen Wu, Shuai Hou, Zhengyu Qing, Zhuo Zheng, Danning Ke, Qihong Lin, Qiang Wang, Shaohuai Shi, Xiaowen Chu
arXiv:2609.40127v1 Announce Type: cross
Abstract: Modern transformers pair impressive capabilities with substantial memory and compute demands. Low-rank weight factorization reduces both while keepin...
By Massimo Bini, Anders Christensen, Stephan Alaniz, Judah Goldfeder, Ole Winther, Yann LeCun, Ravid Shwartz-Ziv, Zeynep Akata
Large language models (LLMs) exhibit exceptional general language processing capabilities, but their memory and compute costs hinder deployment. Ternarization has emerged as a promising compression technique, offering significant reductions in model size and inference complexity.
arXiv:2602. 06694v3 Announce Type: replace Abstract: Weight-only quantization has become a standard approach for efficiently serving large language models (LLMs).
By Hyochan Chong, Dongkyu Kim, Changdong Kim, Minseop Choi
Modern transformers pair impressive capabilities with substantial memory and compute demands. Low-rank weight factorization reduces both while keeping the matrices dense, and thus efficient on standar...