arXiv Machine Learning

When More Parameters Hurt: Foundation Model Priors Amplify Worst-Client Disparity Under Extreme Federated Heterogeneity

arXiv AI
Sep 2

Breaking the Structural Identity: Personalized Federated LoRA Fine-tuning under Rank Heterogeneity

The paper introduces FedRoRA, a federated learning framework that combines Low‑Rank Adaptation (LoRA) with rank‑heterogeneous personalization. It separates model adaptation into shared global directions and client‑specific rank‑wise magnitudes, using SVD on the server to extract a global subspace and a personalized projection with top‑k selection for each client. Experiments on natural language understanding and generation tasks show that FedRoRA outperforms existing state‑of‑the‑art methods.

By Lei Wang, Jieming Bian, Letian Zhang, Jie Xu
Hugging Face Trending Papers
Aug 10

FedTVD: Balancing Data Quality and Quantity for Robust Federated Learning

Federated Learning (FL) enables collaborative model training across distributed client devices while preserving data privacy. However, FL faces significant challenges due to data heterogeneity, particularly in terms of label distribution skewness and variations in dataset sizes, which can lead to biased model updates and hinder convergence.

arXiv AI
Aug 25

FedCC: Towards Addressing Label Distribution Skews in Distillation-Based Federated Learning

FedCC is a new algorithm for distillation-based federated learning that tackles label distribution skew by allowing clients to mark ambiguous samples as 'unknown' instead of forcing a potentially wrong classification. By adding this extra class and calibrating pseudo-labels on a public dataset, FedCC balances confidence across majority and minority classes. Experiments show that FedCC outperforms existing methods, achieving 67.3% accuracy even when each client has data from only one of ten classes, whereas baselines drop to near-random performance.

By Wenxuan Ye, Onur Ayan, Xueli An, Georg Carle