arXiv:2602. 23638v3 Announce Type: replace-cross Abstract: Federated LoRA provides a communication-efficient mechanism for fine-tuning large language models on decentralized data.
arXiv:2608. 15381v1 Announce Type: new Abstract: Low-Rank Adaptation (LoRA) enables efficient federated fine-tuning of large language models, but its factorized parameterization creates a tension between accurate aggregation of local updates and continuity of locally optimized factors.
Low-Rank Adaptation (LoRA) enables efficient federated fine-tuning of large language models, but its factorized parameterization creates a tension between accurate aggregation of local updates and con...
arXiv:2608. 03605v1 Announce Type: new Abstract: Federated fine-tuning with Low-Rank Adaptation (LoRA) enables efficient collaborative adaptation of Large Language Models (LLMs) without centralizing private data.
arXiv:2608. 09742v1 Announce Type: cross Abstract: Low-rank adaptation (LoRA) represents large language model (LLM) updates with two compact matrix factors, i.
arXiv:2606. 15963v1 Announce Type: cross Abstract: Federated fine-tuning of large language models using parameter-efficient methods such as LoRA enables privacy-preserving adaptation of foundation models.
The paper introduces FedRoRA, a federated learning framework that combines Low‑Rank Adaptation (LoRA) with rank‑heterogeneous personalization. It separates model adaptation into shared global directions and client‑specific rank‑wise magnitudes, using SVD on the server to extract a global subspace and a personalized projection with top‑k selection for each client. Experiments on natural language understanding and generation tasks show that FedRoRA outperforms existing state‑of‑the‑art methods.
arXiv:2609.36986v1 Announce Type: new Abstract: Federated LoRA fine-tuning enables parameter-efficient adaptation of pre-trained models without sharing private data, but suffers from two fundamental...
arXiv:2601. 11219v3 Announce Type: replace-cross Abstract: Federated learning (FL) for large language models (LLMs) has attracted increasing attention as a privacy-preserving approach for adapting models over distributed data, where parameter-efficient methods such as Low-Rank Adaptation (LoRA) are widely adopted to reduce communication and memory costs.
arXiv:2606. 06154v1 Announce Type: new Abstract: Federated fine-tuning of foundation models using Low-Rank Adaptation (LoRA) offers a communication efficient solution for distributed learning.
arXiv:2607. 22251v1 Announce Type: new Abstract: Low-Rank Adaptation (LoRA) is a widely used parameter-efficient fine-tuning method for large language models, but its performance depends strongly on how a fixed rank budget is distributed across Transformer modules.
arXiv:2608. 10144v1 Announce Type: new Abstract: We consider federated parameter efficient fine-tuning of large neural networks with low-rank adaptation (LoRA,~Hu et al.
Federated fine-tuning of large pre-trained models increasingly relies on Low-Rank Adaptation (LoRA) to reduce communication and computation, but heterogeneous clients can make adapter aggregation unstable. We identify the data-parameter interference as a geometric source of this instability.