arXiv Machine Learning June 19, 2026 By Nico Harder, Daniel Becking, Karsten Mueller, Wojciech Samek

Activation- and Influence-Aware Ranks (AIR): Function-Preserving SVD Compression for LLMs

Read the original on arXiv Machine Learning →

arXiv:2606. 19993v1 Announce Type: new Abstract: We present Activation- and Influence-Aware Ranks (AIR), an SVD-based LLM compression framework that guides each weight matrix's low-rank approximation with a backward-signal influence metric.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 7

LACE-SVD: Loss-Aware SVD with Cumulative Error Correction for LLM Compression

arXiv:2607. 03057v1 Announce Type: cross Abstract: The rapid growth in the parameter scale of large language models (LLMs) has created a strong demand for efficient compression techniques.

By Zhuowen Liu, Longkun Hao, Shiyu Feng, Xiaowen Chang, Ruiqun Li, Changqun Li

llms efficiency

arXiv Machine Learning

Jun 8

SigmaScale: LLM Compression with SVD-based Low-Rank Decomposition and Learned Scaling Matrices

arXiv:2606. 07098v1 Announce Type: cross Abstract: We present SigmaScale, a method for learning auxiliary scaling matrices $S$ to aid truncated Singular Value Decomposition (SVD) based Large Language Model (LLM) compression.

By Ernests Lavrinovics, Marco Letizia, Roy Janco, Shai Segal, Johannes Bjerva, Maurizio Pierini

llms benchmarks

arXiv AI

Aug 11

Understanding Calibration and Truncation Error Propagation in Training-Free Low-Rank Compression for LLMs

arXiv:2608. 08506v1 Announce Type: new Abstract: Training-free low-rank compression frameworks have been gaining prominence for LLM compression given their effectiveness in reducing model parameter count while maintaining task-level accuracy.

By Mohanad Odema, Gabrielle De Micheli, Dayin Gou, Nilesh Malpeddi, Prathamesh Vaste, Jacob Song

llms benchmarks

arXiv Machine Learning

Jun 9

Cross-Layer Subspace Coupling for LLM Compression: A Unifying Framework and Its Empirical Limits

arXiv:2605. 30836v2 Announce Type: replace Abstract: Recent SVD based compression methods for large language models like SVD LLM and Basis Sharing can be unified under one optimization problem.

By Snigdha Chandan Khilar

llms

arXiv Machine Learning

Jun 5

Activation-Informed Pareto-Guided Low-Rank Compression for Efficient LLM/VLM

arXiv:2510. 05544v2 Announce Type: replace-cross Abstract: Large language models (LLM) and vision-language models (VLM) have achieved state-of-the-art performance, but they impose significant memory and computing challenges in deployment.

By Ryan Solgi, Parsa Madinei, Jiayi Tian, Rupak Swaminathan, Jing Liu, Nathan Susanj, Zheng Zhang

llms efficiency multimodal benchmarks

arXiv Machine Learning

Jul 3

BALF: Budgeted Activation-Aware Low-Rank Factorization for Fine-Tuning-Free Model Compression

arXiv:2509. 25136v3 Announce Type: replace Abstract: Activation-aware low-rank factorization techniques yield strong compression results but are generally confined to linear layers, while existing whitening-based theory typically makes an implicit full-rank assumption on activations.

By David Gonz\'alez-Mart\'inez

llms fine-tuning efficiency

Related stories

LACE-SVD: Loss-Aware SVD with Cumulative Error Correction for LLM Compression

SigmaScale: LLM Compression with SVD-based Low-Rank Decomposition and Learned Scaling Matrices

Understanding Calibration and Truncation Error Propagation in Training-Free Low-Rank Compression for LLMs

Cross-Layer Subspace Coupling for LLM Compression: A Unifying Framework and Its Empirical Limits

Activation-Informed Pareto-Guided Low-Rank Compression for Efficient LLM/VLM

BALF: Budgeted Activation-Aware Low-Rank Factorization for Fine-Tuning-Free Model Compression