← Back to all news
arXiv Machine Learning By Dmitriy Bystrov, George Chirkov, Daniil Medyakov, Dmitry Bylinkin, Aleksandr Beznosikov

Bridging Compressed Communication and Local Steps via Spectral Method in Federated Learning

Read the original on arXiv Machine Learning →

The paper introduces a new spectral federated learning method that combines local updates and communication compression, addressing key bottlenecks in FL such as data heterogeneity and high communication overhead. It builds on the spectral optimizer Muon, which has shown superiority over Adam-like methods in centralized training, and adapts it to the federated setting. The authors provide rigorous convergence guarantees without requiring the restrictive bounded heterogeneity assumption and validate their approach on standard image classification benchmarks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Jun 16

Photon: Federated LLM Pre-Training

arXiv:2411. 02908v2 Announce Type: replace Abstract: Scaling large language models (LLMs) demands extensive data and computing resources, which are traditionally constrained to data centers by the high-bandwidth requirements of distributed training.

By Lorenzo Sani, Alex Iacob, Zeyu Cao, Royson Lee, Bill Marino, Yan Gao, Dongqi Cai, Zexi Li, Wanru Zhao, Xinchi Qiu, Nicholas D. Lane
llms
More like this →
arXiv Machine Learning
Oct 3

FedSAP: Federated Learning with Structured Adaptive Partitioning for Multi-Domain Heterogeneous Edge Devices

FedSAP is a federated learning framework that addresses heterogeneous edge devices by using structured pruning as a budget-constrained tri-state channel allocation. It partitions model channels into a Global pool, pseudo-domain-specific Private pools, and a Dropped state, allowing broadly useful features to be shared while isolating domain-sensitive updates. Experiments on Digits and Office-Caltech datasets show FedSAP achieving higher mean global accuracy than the strongest baseline while supporting up to 80% client pruning ratios.

By Wentao Yue, Tianyou Lai, Hongji Li, Qingyu Mao, Qilei Li
efficiency
More like this →
arXiv Machine Learning
Sep 11

MUC-FL: Block-Wise Marginal Utility Contribution for Communication-Efficient Federated Learning

The paper introduces MUC-FL, a block‑wise marginal utility contribution framework that selectively transmits only the most impactful data blocks in federated learning to reduce communication overhead. Applied to a multimodal dataset derived from multiple MIMIC clinical datasets, the method identifies 24 out of 1,135 candidate blocks (1.76%) as carrying meaningful improvement signals, potentially cutting communication by 45‑50% while preserving or enhancing model quality. The deduplication‑based block selection achieves a macro F1 score of 0.8566 versus 0.8155 for standard federated optimization, showing improved performance especially for underrepresented classes.

By Akshay Mhatre, Vikram Karthick, Deepti Gupta, Jia Zou
multimodal
More like this →