Modern transformers pair impressive capabilities with substantial memory and compute demands. Low-rank weight factorization reduces both while keeping the matrices dense, and thus efficient on standar...
arXiv:2606. 02559v1 Announce Type: cross Abstract: Post-training compression of Large Language Models (LLMs) removes entire architectural components, either deleting them or replacing them with fitted modules.
By Elia Cunegatti, Marcus Vukojevic, Erik Nielsen, Giovanni Iacca
arXiv:2607.12550v3 Announce Type: replace-cross
Abstract: The key-value (KV) cache has become the dominant memory cost of transformer inference: it grows with batch size, context length, and depth, a...
By Rahul Krishnan, Volker Schulz
arXiv:2610.32009v1 Announce Type: new
Abstract: Mixture-of-Experts (MoE) models reduce computation through sparse activation, but storing their expert parameters remains a major memory bottleneck. We...
By Carson Cary, Yukai Jin, Yihao Huang, Zhankai Ye, Fangyi Li, Ting Hua, Xin Liu, Shangqian Gao
arXiv:2609.06341v1 Announce Type: cross
Abstract: Linear algebra provides the framework of concepts (matrix rank, singular value decomposition (SVD), and eigendecomposition) that modern artificial in...
By Anjaneya Teja Sarma Kalvakolanu
arXiv:2411. 09816v5 Announce Type: replace Abstract: Large neural networks achieve state-of-the-art performance on many tasks, yet their sheer size hinders deployment on resource-constrained devices.
By Cem \"Uy\"uk, Mike Lasby, Mohamed Yassin, Utku Evci, Yani Ioannou
Activation-Aware Weight Tensorization (AWT) is a training‑free calibration wrapper that preconditions each weight matrix with a diagonal activation‑derived scale before applying standard Tensor Train (TT) or Tree Tensor Network (TTN) decompositions. By adding only an input‑side elementwise rescaling, AWT improves functional fidelity of post‑training tensor‑network compression across several large language models (Llama 3.1 8B, Ministral 8B, Qwen2.5 7B) at 2–6× compression, closing a significant portion of the perplexity gap to dense baselines and enhancing downstream task performance. The method demonstrates that diagonal preconditioning offers a robustness‑modularity trade‑off, outperforming full‑covariance oracles in many cases while remaining modular and solver‑agnostic.
By Alessandro Beatini, Marco Maronese, Emanuele Rodol\`a
arXiv:2607. 12550v1 Announce Type: new Abstract: The key-value (KV) cache has become the dominant memory cost of transformer inference.
By Rahul Krishnan, Volker Schulz
COMPOT is a training‑free compression framework for transformer models that uses a small calibration dataset to estimate a sparse weight factorization. It employs orthogonal dictionaries, enabling closed‑form Procrustes updates and analytical single‑step sparse coding, which dramatically reduces optimization cost. The method also introduces a one‑shot dynamic allocation strategy to adaptively redistribute compression rates across layers, achieving a superior quality‑compression trade‑off compared to strong low‑rank and sparse baselines while remaining compatible with post‑training quantization.
By Denis Makhov, Dmitriy Shopkhoev, Magauiya Zhussip, Ammar Ali, Baher Mohammad, Stamatios Lefkimmiatis
BellowsKV introduces a training‑free approach that maps the hidden states of multiple adjacent layers in a large language model to a single shared latent representation per token, allowing each layer to retrieve its own keys and values via a layer‑specific projection. The method retains the latent at full dimension and offers two compression strategies: BellowsKV‑Quant, which stores the latent at low precision, and BellowsKV‑Evict, which makes a shared eviction decision per block. Experiments on Llama‑3.1‑8B, Mistral‑7B, and Qwen2.5‑7B show that BellowsKV‑Quant preserves about 99% of full‑cache accuracy at 3.6× compression and 97‑98% at 13‑14×, outperforming existing cross‑layer methods that drop to at most 84% accuracy at 16× compression.
By Jiamu Zhang, Hanjie Chen
arXiv:2607. 15456v1 Announce Type: new Abstract: Looped, weight-tied Transformers reduce parameters by reusing a block, but decoding still stores a separate K/V cache for every recurrence step.
By James O' Neill, Fergal Reid
DictLLM is a post‑training compression method that reduces the memory footprint of large language models by decomposing weight matrices into shared dictionaries and sparse layer‑specific coefficients. The approach involves four steps: extracting shared dictionaries via SVD, sparsifying coefficients with a dictionary‑aware Hessian, refining dictionaries, and aggressively quantizing both components using Hessian information. Experiments show that DictLLM outperforms existing low‑rank compression techniques, achieving a 5.1× compression of LLaMA‑7B with only a 2.9‑point increase in WikiText‑2 perplexity, and a 3.6× higher compression ratio than the state‑of‑the‑art method.
By Jinho Park, Se Young Chun, Mingoo Seok