arXiv:2608. 03605v1 Announce Type: new Abstract: Federated fine-tuning with Low-Rank Adaptation (LoRA) enables efficient collaborative adaptation of Large Language Models (LLMs) without centralizing private data.
By Shenghui Li, Thiemo Voigt
arXiv:2510. 00192v3 Announce Type: replace Abstract: Low-rank adaptation (LoRA) has become a widely used paradigm for parameter-efficient fine-tuning of large language models, yet its representational capacity often lags behind full fine-tuning.
By Xin Yu, Cong Xie, Xunmei Liu, Tiantian Fan, Lingzhou Xue, Zhi Zhang
The paper introduces ρ_p-LoRA, a rank-allocation strategy for low-rank adaptation (LoRA) in large language models that uses ρ_p regularization (0 < p < 1) to induce sparsity in rank-one components. By regularizing the energy of each component, redundant parts are encouraged to vanish while important ones are retained, leading to an implicit thresholding criterion derived from a two-dimensional proximal subproblem. Experiments on natural language understanding and question-answering tasks show that ρ_p-LoRA achieves performance comparable to existing LoRA baselines.
By Zebang Xie, Chuanyang Zheng, Yik-Chung Wu, Yihang Gao
The paper introduces αp-LoRA, a rank-allocation strategy for low-rank adaptation (LoRA) in large language models that uses π-regularization (0 < p < 1) to induce sparsity in rank-one components. By regularizing the energy of each component, redundant parts are encouraged to vanish while important ones are retained, and the authors derive a proximal subproblem that reduces the matrix optimization to a two‑dimensional thresholding criterion. Experiments on natural language understanding and question‑answering tasks show that αp-LoRA achieves performance competitive with existing LoRA baselines.
arXiv:2609.05885v1 Announce Type: new
Abstract: Low-rank adaptation (LoRA) has become the standard for parameter-efficient fine-tuning of large language models. Most LoRA variants follow a uniform-LR...
By Huiyi Wang, Daijiao Liu, Lina Yao, Dong Gong
arXiv:2607. 20205v1 Announce Type: cross Abstract: Low-rank adaptation (LoRA) has become a widely used parameter-efficient fine-tuning method for large language models.
By Yihang Gao, Vincent Y. F. Tan