arXiv AI

FedFiOT: Fisher-Weighted Optimal Transport for Model Merging in One-Shot and Few-Round Federated Learning

FedFiOT introduces a Fisher-weighted optimal transport framework for merging client models in one-shot and few-round federated learning. By using Fisher information to assess parameter importance, it aligns mismatched neurons and weights parameters during aggregation. The method alternates between optimizing neuron correspondences and global model parameters, and theoretical analysis shows monotonic improvement of the merging objective with a bound on heterogeneity effects. Experiments on multiple datasets confirm consistent performance gains over existing baselines.

arXiv Machine Learning
Sep 18

FedFIbOS: Fisher Importance based Optimal Submodelling for Heterogeneous Federated Learning

FedFIbOS introduces a Fisher‑importance based criterion for selecting submodel parameters in heterogeneous federated learning, addressing the lack of theoretical justification in prior heuristic methods. By deriving a Fisher‑weighted quadratic masking surrogate and showing that the raw Fisher top‑k rule satisfies this surrogate under a Fisher‑dominant ranking condition, the method preserves convergence guarantees while efficiently estimating Fisher scores from squared gradients. Experiments on CIFAR‑10, CIFAR‑100, and AGNews demonstrate that FedFIbOS outperforms state‑of‑the‑art approaches by roughly 10% in accuracy, especially under strong non‑IID heterogeneity.

By Yasmeen Afzal, Jeremiah D. Deng, Haibo Zhang
arXiv Machine Learning
Jun 16

Distribution Alignment for One-Shot Federated Learning via Optimal Transport

arXiv:2606. 16655v1 Announce Type: new Abstract: One-Shot Federated Learning (OSFL) addresses extreme communication regimes in which clients interact with the server only once, amplifying the impact of heterogeneous client data distributions.

By Daniele Berardini (AI for Good), Vito Paolo Pastore (AI for Good, MaLGa-DIBRIS, University of Genoa, Genoa, Italy), Vittorio Murino (AI for Good, Department of Computer Science, University of Verona, Verona, Italy)
arXiv Machine Learning
Aug 31

Beyond Non-IID: Learner--Client Distribution Mismatch in Federated Learning

The paper addresses the mismatch between learner and client data distributions in federated learning, noting that traditional client selection methods often ignore this misalignment. It introduces a dynamic, influence-aware client selection framework that uses a small proxy dataset to estimate each client's utility for the learner’s objective, prioritizing informative sources while mitigating noise and heterogeneity. Experiments on CIFAR-10 with heterogeneous partitions show the proposed method outperforms static and dynamic baselines, achieving faster convergence and higher accuracy.

By Yiming Xie, Lili Su, Ningfang Mi
arXiv Machine Learning
Aug 27

Differentiated Aggregation to Improve Generalization in Federated Learning

The paper proposes a new federated learning approach called FedALS that reduces communication costs by varying aggregation frequencies across model layers. It derives tighter generalization bounds for one‑round and multi‑round federated learning, linking these bounds to local updates and data heterogeneity. Based on representation‑learning insights, the authors argue that infrequent aggregation of early layers and more frequent aggregation of final layers yields more generalizable models, especially in non‑iid settings, and demonstrate the method’s effectiveness experimentally.

By Peyman Gholami, Hulya Seferoglu
arXiv Machine Learning
Oct 3

From Task Mixtures to Specialized Experts

The paper investigates federated learning where each client’s data consists of unknown mixtures of distinct tasks, a scenario termed compound heterogeneity. It shows that when tasks share a common feature geometry, the optimal model for a mixed client is a convex combination of task‑specific models, motivating input‑dependent routing to specialized experts. The authors propose FedSEE, a method that recovers task experts via a convex program and achieves better performance than baselines, reducing negative transfer by 2.9 points overall and 3.7 points for the worst‑served quartile.

By Hojat Allah Salehi, Mehrdad Mahdavi, Andrew Arash Mahyari, M. Hadi Amini
arXiv AI
Aug 11

Multimodal Federated Learning under Dual-Axis Modality Missingness

arXiv:2608. 09240v1 Announce Type: cross Abstract: Multimodal federated learning (FL) supports collaborative modeling in privacy-sensitive health-sensing and medical settings, but realistic deployments often exhibit dual-axis modality missingness: clients have different modality sets, and individual samples may contain only subsets of the modalities available locally.

By Adiba Orzikulova, Jaehyun Kwak, Jaemin Shin, Yunqi Guo, Xiaomin Ouyang, Guoliang Xing, Steven Euijong Whang, Sung-Ju Lee