arXiv Machine Learning July 17, 2026 By Haobo Zhang, Jiankun Wang, Suraj Rajendran, Weishen Pan, Lam Tsoi, Yong Chen, Fei Wang, Jiayu Zhou

Dysco: Dynamic Subspace Boosting to Mitigate LoRA Interference in Federated Learning

Read the original on arXiv Machine Learning →

arXiv:2607. 14367v1 Announce Type: new Abstract: Federated fine-tuning of large pre-trained models increasingly relies on Low-Rank Adaptation (LoRA) to reduce communication and computation, but heterogeneous clients can make adapter aggregation unstable.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

Hugging Face Trending Papers

Jul 15

Dysco: Dynamic Subspace Boosting to Mitigate LoRA Interference in Federated Learning

Federated fine-tuning of large pre-trained models increasingly relies on Low-Rank Adaptation (LoRA) to reduce communication and computation, but heterogeneous clients can make adapter aggregation unstable. We identify the data-parameter interference as a geometric source of this instability.

llms fine-tuning safety

arXiv AI

Aug 11

Rethinking Factor Sharing in Federated LoRA: A Rank-Aware Adaptive Approach

arXiv:2608. 09742v1 Announce Type: cross Abstract: Low-rank adaptation (LoRA) represents large language model (LLM) updates with two compact matrix factors, i.

By Xinyi Xu, Bingnan Xiao, Shuang Qin, Gang Feng, Tony Q. S. Quek

llms fine-tuning

arXiv AI

Jun 6

Amortizing Federated Adaptation: Hypernetwork Driven LoRA for Personalized Foundation Models

arXiv:2606. 06154v1 Announce Type: new Abstract: Federated fine-tuning of foundation models using Low-Rank Adaptation (LoRA) offers a communication efficient solution for distributed learning.

By Sunny Gupta, Shambhavi Shanker, Amit Sethi

fine-tuning multimodal benchmarks safety

arXiv Machine Learning

Jul 24

Spectral Transformation for Layer-wise Global Rank Discovery in Federated LoRA for Vision Transformers

arXiv:2607. 21074v1 Announce Type: new Abstract: Fine-tuning Vision Transformers (ViTs) with low-rank adapters (LoRA) promises better communication efficiency under federated setup, yet existing aggregation strategies face fundamental limitations.

By Hariharan Ramesh, Jyotikrishna Dass

llms diffusion fine-tuning

arXiv Machine Learning

Jul 7

Learning Only What Valid Adapters Can Express: Subspace-Constrained Adaptation Against Fine-Tuning Poisoning

arXiv:2607. 05300v1 Announce Type: new Abstract: Parameter-efficient fine-tuning still leaves a broad space of behavior-changing updates reachable, so a poisoned objective can be represented and optimized.

By Fabien Polly

fine-tuning

arXiv AI

Jun 15

FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA

arXiv:2602. 23638v3 Announce Type: replace-cross Abstract: Federated LoRA provides a communication-efficient mechanism for fine-tuning large language models on decentralized data.

By Haoran Zhang, Dongjun Kim, Seohyeon Cha, Haris Vikalo

llms fine-tuning safety

Related stories

Dysco: Dynamic Subspace Boosting to Mitigate LoRA Interference in Federated Learning

Rethinking Factor Sharing in Federated LoRA: A Rank-Aware Adaptive Approach

Amortizing Federated Adaptation: Hypernetwork Driven LoRA for Personalized Foundation Models

Spectral Transformation for Layer-wise Global Rank Discovery in Federated LoRA for Vision Transformers

Learning Only What Valid Adapters Can Express: Subspace-Constrained Adaptation Against Fine-Tuning Poisoning

FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA