The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
The paper introduces the "Compression Trinity," a unified framework that jointly applies sparsity, quantization, and low‑rank approximations to compress large language models. It presents several methods—MKOR, SLoPe, OPTIMA, PATCH, and SLiM—that leverage these three pillars to accelerate training, reduce memory bandwidth, and recover accuracy, achieving significant speedups and accuracy gains over existing techniques. The results demonstrate that combining all three compression strategies is essential for efficient, scalable, high‑performance LLM deployment.
arXiv:2601. 16991v3 Announce Type: replace-cross Abstract: Adapting large pre-trained language models to downstream tasks often entails fine-tuning millions of parameters or deploying costly dense weight updates, which hinders their use in resource-constrained environments.
The paper introduces 3BASiL-TM, a one‑shot post‑training method for compressing large language models using a sparse plus low‑rank decomposition. It employs a novel 3‑block ADMM algorithm to minimize layer‑wise reconstruction error and a transformer‑matching refinement that jointly optimizes sparse and low‑rank components across layers. Experiments show a >30% reduction in WikiText2 perplexity gap for LLaMA‑8B under a (2:4 sparse + 64 LR) setting and a 2.5× faster compression runtime on an A100 GPU compared to state‑of‑the‑art methods.
arXiv:2606. 26587v1 Announce Type: cross Abstract: Low-bit floating-point formats and semi-structured sparsity are increasingly supported by modern accelerators, yet combining them for LLM activation compression remains challenging: activations contain input-dependent outliers that dominate block scales in FP4 quantization, and directly applying N:M sparsity masks discards moderate values, coupling sparsification loss with quantization error.
arXiv:2610.04204v2 Announce Type: replace Abstract: Training large language models (LLMs) entails a fundamental trade-off: memory-efficient optimizers such as Adam discard cross-parameter curvature,...
Semi-structured 2:4 sparsity is widely supported by modern accelerators, providing up to a 2x theoretical speedup. However, its strict 50% sparsity constraint often causes non-negligible accuracy degradation under post-training pruning.