arXiv Machine Learning

Coordination on a Budget: Federated Active Learning with Few Labels

The paper introduces a federated active learning (FAL) approach that tackles data privacy and label scarcity by coordinating query selection across clients. In low-budget scenarios, it finds that homogeneous (IID) data actually requires stronger coordination to avoid redundant queries, while heterogeneous data naturally yields diversity—a reversal of the usual federated learning narrative. The authors propose a new framework that aligns client data in a shared embedding space via federated representation learning, enabling globally coordinated active selection while keeping annotations local, and demonstrate that this method outperforms existing FAL methods even with larger annotation budgets.

arXiv AI
Aug 17

Federated Prompt Learning: A Unified Framework, Empirical Analysis, and Future Directions

arXiv:2608. 13844v1 Announce Type: cross Abstract: Large language models (LLMs) have become core components of cloud-based intelligent services in academia and industry, yet their training and deployment are hindered by high computational costs, data centralization, and privacy concerns.

By Qinglin Yang, Chen Qiu, Hongyuan Zhang, Pengdeng Li, Yuan Liu, Zhihong Tian
arXiv Machine Learning
Jun 9

Federated Large Language Models: Current Progress and Future Directions

arXiv:2409. 15723v3 Announce Type: replace Abstract: Large Language Models have achieved impressive performance across diverse applications, yet their training typically depends on centralized data collection, raising serious privacy and governance concerns.

By Yuhang Yao, Jianyi Zhang, Junda Wu, Chengkai Huang, Yu Xia, Tong Yu, Ruiyi Zhang, Sungchul Kim, Ryan Rossi, Ang Li, Lina Yao, Julian McAuley, Yiran Chen, Carlee Joe-Wong
arXiv AI
2d ago

Breaking the Structural Identity: Personalized Federated LoRA Fine-tuning under Rank Heterogeneity

The paper introduces FedRoRA, a federated learning framework that combines Low‑Rank Adaptation (LoRA) with rank‑heterogeneous personalization. It separates model adaptation into shared global directions and client‑specific rank‑wise magnitudes, using SVD on the server to extract a global subspace and a personalized projection with top‑k selection for each client. Experiments on natural language understanding and generation tasks show that FedRoRA outperforms existing state‑of‑the‑art methods.

By Lei Wang, Jieming Bian, Letian Zhang, Jie Xu
Hugging Face Trending Papers
Aug 10

FedTVD: Balancing Data Quality and Quantity for Robust Federated Learning

Federated Learning (FL) enables collaborative model training across distributed client devices while preserving data privacy. However, FL faces significant challenges due to data heterogeneity, particularly in terms of label distribution skewness and variations in dataset sizes, which can lead to biased model updates and hinder convergence.