arXiv AI July 17, 2026 By Viktoriia Chekalina, Daniil Moskovskiy, Tatiana Matveeva, Andrey Kuznetsov, Evgeny Frolov

Generalized Fisher-Weighted SVD: Scalable Kronecker-Factored Fisher Approximation for Compressing Large Language Models

arXiv:2505. 17974v2 Announce Type: replace-cross Abstract: The Fisher information is a fundamental concept for characterizing the sensitivity of parameters in neural networks.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv Machine Learning

Jun 8

SigmaScale: LLM Compression with SVD-based Low-Rank Decomposition and Learned Scaling Matrices

arXiv:2606. 07098v1 Announce Type: cross Abstract: We present SigmaScale, a method for learning auxiliary scaling matrices $S$ to aid truncated Singular Value Decomposition (SVD) based Large Language Model (LLM) compression.

By Ernests Lavrinovics, Marco Letizia, Roy Janco, Shai Segal, Johannes Bjerva, Maurizio Pierini

llms benchmarks

arXiv AI

Jul 22

Compressing What Matters: Neuron Importance Meets Data-Aware Low Rank Approximation for Language Model Compression

arXiv:2607. 18284v1 Announce Type: cross Abstract: To excel at their domain large language models are comprised of billions of parameters.

By Athanasios Ntovas, Alexandros Doumanoglou, Petros Drakoulis, Dimitris Zarpalas

llms efficiency benchmarks

arXiv AI

Jun 6

LLMCodec: Adapting Video Codecs for Efficient Weight Compression of Large Language Models

arXiv:2606. 05861v1 Announce Type: cross Abstract: The rapid development of large language models(LLMs) has led to remarkable advances in natural language processing.

By Rui Wang, Yan Zhao, Li Song, Zhengxue Cheng

llms nlp fine-tuning efficiency

arXiv Machine Learning

Jul 17

Stabilizing Native Low-Rank LLM Pretraining

arXiv:2602. 12429v2 Announce Type: replace Abstract: Foundation models have achieved remarkable success, yet their growing parameter counts pose significant computational and memory challenges.

By Paul Janson, Edouard Oyallon, Eugene Belilovsky

llms rag

arXiv Machine Learning

Jun 2

LASER: Loss-Aware Singular-value Decomposition and Rank Allocation for Efficient Low-Precision Vision-Language Models

arXiv:2606. 00573v1 Announce Type: new Abstract: Vision-language models (VLMs) deliver strong multimodal reasoning capabilities, but their large computational cost and high parameter counts make deployment challenging on resource-constrained devices.

By Haiyu Wang, Yutong Wang, Leshu Li, Yihui Ren, Sai Qian Zhang

llms efficiency multimodal

arXiv AI

Jul 7

LACE-SVD: Loss-Aware SVD with Cumulative Error Correction for LLM Compression

arXiv:2607. 03057v1 Announce Type: cross Abstract: The rapid growth in the parameter scale of large language models (LLMs) has created a strong demand for efficient compression techniques.

By Zhuowen Liu, Longkun Hao, Shiyu Feng, Xiaowen Chang, Ruiqun Li, Changqun Li

llms efficiency

Related stories

SigmaScale: LLM Compression with SVD-based Low-Rank Decomposition and Learned Scaling Matrices

Compressing What Matters: Neuron Importance Meets Data-Aware Low Rank Approximation for Language Model Compression

LLMCodec: Adapting Video Codecs for Efficient Weight Compression of Large Language Models

Stabilizing Native Low-Rank LLM Pretraining

LASER: Loss-Aware Singular-value Decomposition and Rank Allocation for Efficient Low-Precision Vision-Language Models

LACE-SVD: Loss-Aware SVD with Cumulative Error Correction for LLM Compression