arXiv Computation and Language

Learning Functional Subspaces for Neural Network Compression

arXiv Machine Learning
Oct 8

Activation-Aware Weight Tensorization: A Calibration-Time Preconditioner for Tensor-Network LLM Compression

Activation-Aware Weight Tensorization (AWT) is a training‑free calibration wrapper that preconditions each weight matrix with a diagonal activation‑derived scale before applying standard Tensor Train (TT) or Tree Tensor Network (TTN) decompositions. By adding only an input‑side elementwise rescaling, AWT improves functional fidelity of post‑training tensor‑network compression across several large language models (Llama 3.1 8B, Ministral 8B, Qwen2.5 7B) at 2–6× compression, closing a significant portion of the perplexity gap to dense baselines and enhancing downstream task performance. The method demonstrates that diagonal preconditioning offers a robustness‑modularity trade‑off, outperforming full‑covariance oracles in many cases while remaining modular and solver‑agnostic.

By Alessandro Beatini, Marco Maronese, Emanuele Rodol\`a
arXiv Machine Learning
Oct 28

COMPOT: Calibration-Optimized Matrix Procrustes Orthogonalization for Transformers Compression

COMPOT is a training‑free compression framework for transformer models that uses a small calibration dataset to estimate a sparse weight factorization. It employs orthogonal dictionaries, enabling closed‑form Procrustes updates and analytical single‑step sparse coding, which dramatically reduces optimization cost. The method also introduces a one‑shot dynamic allocation strategy to adaptively redistribute compression rates across layers, achieving a superior quality‑compression trade‑off compared to strong low‑rank and sparse baselines while remaining compatible with post‑training quantization.

By Denis Makhov, Dmitriy Shopkhoev, Magauiya Zhussip, Ammar Ali, Baher Mohammad, Stamatios Lefkimmiatis
arXiv Machine Learning
Oct 28

BellowsKV: One Latent for Many Layers, Expanded on Demand

BellowsKV introduces a training‑free approach that maps the hidden states of multiple adjacent layers in a large language model to a single shared latent representation per token, allowing each layer to retrieve its own keys and values via a layer‑specific projection. The method retains the latent at full dimension and offers two compression strategies: BellowsKV‑Quant, which stores the latent at low precision, and BellowsKV‑Evict, which makes a shared eviction decision per block. Experiments on Llama‑3.1‑8B, Mistral‑7B, and Qwen2.5‑7B show that BellowsKV‑Quant preserves about 99% of full‑cache accuracy at 3.6× compression and 97‑98% at 13‑14×, outperforming existing cross‑layer methods that drop to at most 84% accuracy at 16× compression.

By Jiamu Zhang, Hanjie Chen
arXiv AI
Nov 6

DictLLM: Post-training Compression of Large Language Model with Dictionary Kernels

DictLLM is a post‑training compression method that reduces the memory footprint of large language models by decomposing weight matrices into shared dictionaries and sparse layer‑specific coefficients. The approach involves four steps: extracting shared dictionaries via SVD, sparsifying coefficients with a dictionary‑aware Hessian, refining dictionaries, and aggressively quantizing both components using Hessian information. Experiments show that DictLLM outperforms existing low‑rank compression techniques, achieving a 5.1× compression of LLaMA‑7B with only a 2.9‑point increase in WikiText‑2 perplexity, and a 3.6× higher compression ratio than the state‑of‑the‑art method.

By Jinho Park, Se Young Chun, Mingoo Seok