arXiv AI July 21, 2026 By Longteng Zhang, Sen Wu, Shuai Hou, Zhengyu Qing, Zhuo Zheng, Danning Ke, Qihong Lin, Qiang Wang, Shaohuai Shi, Xiaowen Chu

Sparsity-Aware Low-Rank Representation for Efficient Fine-Tuning of Large Language Models

Read the original on arXiv AI →

arXiv:2601. 16991v3 Announce Type: replace-cross Abstract: Adapting large pre-trained language models to downstream tasks often entails fine-tuning millions of parameters or deploying costly dense weight updates, which hinders their use in resource-constrained environments.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv Machine Learning

Jul 30

Train Large, Deploy Compact: Structured Compression for Compact Low-Rank Adaptation

arXiv:2510. 00192v3 Announce Type: replace Abstract: Low-rank adaptation (LoRA) has become a widely used paradigm for parameter-efficient fine-tuning of large language models, yet its representational capacity often lags behind full fine-tuning.

By Xin Yu, Cong Xie, Xunmei Liu, Tiantian Fan, Lingzhou Xue, Zhi Zhang

llms fine-tuning efficiency

arXiv AI

Jul 22

Beyond Single-Dimensional Compression: The Compound Sparsity Frontier of Large Language Models

arXiv:2607. 18280v1 Announce Type: cross Abstract: Large language models (LLMs) are often compressed through static parameter pruning or dynamic token-level computation, yet aggressive sparsification can trigger rapid performance degradation beyond an essential sparsity boundary.

By Chao Han, Haozhe Hu, Xiaoyu Shen

llms nlp efficiency benchmarks

arXiv Machine Learning

Jul 13

Super-Tuning: From Activation-Aware Pruning to Sparse Fine-Tuning

arXiv:2607. 09287v1 Announce Type: new Abstract: Large language models (LLMs) remain expensive to fine-tune because full-parameter updates require substantial memory, compute, and per-task storage.

By Ivan Ilin, Philip Zmushko, Peter Richt\'arik

llms fine-tuning efficiency

arXiv Machine Learning

Jul 27

Layer-wise LoRA fine-tuning: a similarity metric approach

arXiv:2602. 05988v2 Announce Type: replace Abstract: Pre-training Large Language Models (LLMs) on web-scale datasets becomes fundamental for advancing general-purpose AI.

By Keith Ando Ogawa, Bruno Lopes Yamamoto, Lucas Lauton de Alcantara, Lucas Pellicer, Rosimeire Pereira Costa, Edson Bollis, Anna Helena Reali Costa, Artur Jordao

llms fine-tuning multimodal benchmarks

arXiv Machine Learning

Jul 14

Prune, Update and Trim: Robust Structured Pruning for Large Language Models

arXiv:2605. 18331v2 Announce Type: replace Abstract: Large Language Models (LLMs) have experienced significant growth and development in recent years.

By Diego Coello de Portugal Mecke, Tom Hanika, Lars Schmidth-Thieme

llms efficiency benchmarks

arXiv Machine Learning

Jun 25

EPTS: Elastic Post-Training Sparsity for Efficient Large Language Model Compression

arXiv:2606. 25285v1 Announce Type: new Abstract: Post-Training Sparsity (PTS) has emerged as a crucial paradigm for compressing Large Language Models to facilitate efficient deployment on resource-constrained devices.

By Ke Xu, Jiaqi Wan, Wenhao Hu, Han Pu, Xiaoyun Wang

llms fine-tuning efficiency benchmarks

Related stories

Train Large, Deploy Compact: Structured Compression for Compact Low-Rank Adaptation

Beyond Single-Dimensional Compression: The Compound Sparsity Frontier of Large Language Models

Super-Tuning: From Activation-Aware Pruning to Sparse Fine-Tuning

Layer-wise LoRA fine-tuning: a similarity metric approach

Prune, Update and Trim: Robust Structured Pruning for Large Language Models

EPTS: Elastic Post-Training Sparsity for Efficient Large Language Model Compression