arXiv Machine Learning

FedRSPO+: A Heterogeneity-aware Algorithm for Decision-focused Federated Learning

arXiv Machine Learning
Oct 3

From Task Mixtures to Specialized Experts

The paper investigates federated learning where each client’s data consists of unknown mixtures of distinct tasks, a scenario termed compound heterogeneity. It shows that when tasks share a common feature geometry, the optimal model for a mixed client is a convex combination of task‑specific models, motivating input‑dependent routing to specialized experts. The authors propose FedSEE, a method that recovers task experts via a convex program and achieves better performance than baselines, reducing negative transfer by 2.9 points overall and 3.7 points for the worst‑served quartile.

By Hojat Allah Salehi, Mehrdad Mahdavi, Andrew Arash Mahyari, M. Hadi Amini
arXiv Machine Learning
1d ago

FedPCD: Priority-Constrained Federated Aggregation

FedPCD introduces a priority-constrained federated optimization framework that decouples the global aggregation objective from client protection. By preserving the standard federated aggregate and applying only minimal corrections to satisfy client performance constraints, FedPCD achieves higher global accuracy and improved worst‑10% client accuracy across heterogeneous benchmarks. The method also maintains competitive trade‑offs between global accuracy and client variability while converging faster with fewer communication rounds.

By Yousef Irshaid, Dara Varam, Mohamed I. AlHajri
arXiv Machine Learning
Oct 3

Latent Information Sharing for Accelerating Federated Learning

The paper introduces a latent information sharing scheme for federated learning that mitigates client drift by sharing a small amount of hidden‑layer activations. The authors demonstrate both theoretically and empirically that this approach improves training efficiency while maintaining convergence guarantees and data privacy. Compared to existing methods such as FedProx, SCAFFOLD, FedPVR, FedProto, and SplitFed, the proposed method achieves higher model accuracy within a fixed round budget without adding significant communication overhead.

By Seungjun Lee, Ensieh Khazaei, Dimitrios Hatzinakos, Baturalp Buyukates, Sunwoo Lee
arXiv Machine Learning
Jun 19

Federated Bilevel Performative Prediction

arXiv:2606. 19734v1 Announce Type: new Abstract: Federated bilevel optimization is widely used for nested learning problems across distributed clients, such as federated hyperparameter tuning and meta-learning under privacy and communication constraints.

By Liangxin Qian, Chang Liu, Xuanyu Cao, Jun Zhao, Kwok-Yan Lam
arXiv Machine Learning
Sep 18

FedFIbOS: Fisher Importance based Optimal Submodelling for Heterogeneous Federated Learning

FedFIbOS introduces a Fisher‑importance based criterion for selecting submodel parameters in heterogeneous federated learning, addressing the lack of theoretical justification in prior heuristic methods. By deriving a Fisher‑weighted quadratic masking surrogate and showing that the raw Fisher top‑k rule satisfies this surrogate under a Fisher‑dominant ranking condition, the method preserves convergence guarantees while efficiently estimating Fisher scores from squared gradients. Experiments on CIFAR‑10, CIFAR‑100, and AGNews demonstrate that FedFIbOS outperforms state‑of‑the‑art approaches by roughly 10% in accuracy, especially under strong non‑IID heterogeneity.

By Yasmeen Afzal, Jeremiah D. Deng, Haibo Zhang
arXiv Machine Learning
Aug 31

Beyond Non-IID: Learner--Client Distribution Mismatch in Federated Learning

The paper addresses the mismatch between learner and client data distributions in federated learning, noting that traditional client selection methods often ignore this misalignment. It introduces a dynamic, influence-aware client selection framework that uses a small proxy dataset to estimate each client's utility for the learner’s objective, prioritizing informative sources while mitigating noise and heterogeneity. Experiments on CIFAR-10 with heterogeneous partitions show the proposed method outperforms static and dynamic baselines, achieving faster convergence and higher accuracy.

By Yiming Xie, Lili Su, Ningfang Mi