The paper proposes two hybrid algorithms, FL+FSDP and FL+HSDP, that combine sharded data parallelism with federated learning-style aggregations to reduce communication overhead in large-scale AI training. By partitioning GPUs into loosely‑coupled federation groups, the methods keep inter‑group traffic minimal while maintaining a bounded global batch size. Experiments on a Llama3.1 8B model trained on 512 A100 GPUs show up to 8.04× faster data processing and 4.48 lower evaluation perplexity compared to conventional sharded DP.
By Gianluca Mittone, Marco Aldinucci
The paper proposes two hybrid algorithms, FL+FSDP and FL+HSDP, that combine sharded data parallelism with federated learning-style aggregations to reduce communication overhead in large-scale AI training. By forming loosely‑coupled federation groups, the methods keep inter‑group traffic minimal while maintaining a bounded global batch size. Experiments on a Llama3.1 8B model trained on 512 A100 GPUs show up to 8.04× faster data processing and 4.48 lower evaluation perplexity compared to traditional sharded DP approaches.
arXiv:2409. 17754v2 Announce Type: replace-cross Abstract: Federated Learning (FL) emerges as a distributed machine learning approach that addresses privacy concerns by training AI models locally on devices.
By Diego Cajaraville-Aboy, Ana Fern\'andez-Vilas, Rebeca P. D\'iaz-Redondo, Manuel Fern\'andez-Veiga
arXiv:2606. 06687v1 Announce Type: new Abstract: We investigate cluster formation, involving the number and composition of clusters, in decentralized federated learning (FL) with heterogeneous machine learning (ML) optimizers.
By Su Wang, Mung Chiang, H. Vincent Poor
StoCFL is a clustered federated learning framework designed to address Non-IID data and dynamic client participation. It introduces a flexible clustering mechanism that allows arbitrary client participation and accommodates newly joined clients, improving data efficiency and model performance. Experiments on four Non-IID settings and a real-world dataset demonstrate that StoCFL achieves promising cluster results even when the number of clusters is unknown, outperforming baseline approaches across various scenarios.
By Dun Zeng, Xiangjing Hu, Shiyu Liu, Yue Yu, Qifan Wang, Zenglin Xu
The paper introduces FastFederatedLearning (FFL), an open‑source federated learning framework written in C/C++ that prioritizes performance and customizability. FFL allows users to define arbitrary communication graphs between clients and servers, and it consistently outperforms Intel OpenFL with speedups of 2.5x to 3.69x across x86‑64, ARM‑v8, and RISC‑V platforms. Future plans include a Python wrapper and middleware for diverse communication backends, aiming to support dynamic, evolving federations as long‑term services.
By Gianluca Mittone, Robert Birke, Marco Aldinucci
arXiv:2607. 28338v1 Announce Type: new Abstract: Clustered Federated Learning (CFL) addresses data heterogeneity in federated settings by grouping clients with similar data distributions to enable effective training.
By Michael Ben Ali, Imen Megdiche, Andr\'e P\'eninou, Olivier Teste
arXiv:2608. 14654v1 Announce Type: cross Abstract: Federated Learning (FL) is a collaborative paradigm that enables multiple devices to train a global model while preserving local data privacy.
By Hai Anh Tran, Cuong Ta, Truong X. Tran
The paper introduces Evolving Federated Density Peak Clustering (eFDPC), a one‑shot federated clustering method that merges evolving fuzzy data‑stream clustering with density peak clustering. Each client independently updates micro‑clusters from private mini‑batches, and only selected density peaks are sent to the server, reducing communication. A shuffler‑based anonymization layer protects client contributions, and two variants—eFDPC‑P and eFDPC‑G—are evaluated on synthetic and real datasets, outperforming recent federated clustering baselines in one‑shot settings and handling high‑dimensional, non‑IID data.
By Miha O\v{z}bot, Seiichi Ozawa, Igor \v{S}krjanc
arXiv:2606. 30499v1 Announce Type: new Abstract: Federated Learning often suffers under non-independently and identically distributed data, where a single global model may fail to represent the diversity of client distributions.
By Davide Domini, Gianluca Aguzzi, Ivana Dusparic, Danilo Pianini, Mirko Viroli
arXiv:2607. 08368v1 Announce Type: new Abstract: With the widespread deployment of basic models in edge intelligence, communication bandwidth has become a core bottleneck restricting the scalability of federated learning.
By Lingyu Qiu, Daniela Annunziata, Stefano Izzo, Fabio Giampaolo, Francesco Piccialli
arXiv:2505.19183v4 Announce Type: replace
Abstract: This book offers a hands-on introduction to building and understanding federated learning (FL) systems. FL enables multiple devices -- such as smar...
By A. Jung