The paper introduces D-FROST, an optimal‑transport‑based algorithm for decentralized federated prompt‑tuning, addressing challenges of heterogeneous local data and non‑IID distributions. It formulates prompt tuning as a Wasserstein optimization over prompt measures and merges neighborhood prompts into compact representative sets. The authors provide theoretical guarantees on consensus error and convergence, and demonstrate effectiveness through experiments on heterogeneous client data.
arXiv:2606. 06687v1 Announce Type: new Abstract: We investigate cluster formation, involving the number and composition of clusters, in decentralized federated learning (FL) with heterogeneous machine learning (ML) optimizers.
arXiv:2610.09091v1 Announce Type: new Abstract: Decision-focused learning (DFL) trains predictive models for downstream optimization, but existing methods largely assume centralized data. In cross-si...
arXiv:2609.21057v1 Announce Type: new Abstract: Federated learning (FL) enables collaborative model training without sharing raw data, but its performance degrades under non-IID data and stochastic c...
arXiv:2505. 11304v4 Announce Type: replace-cross Abstract: Federated learning (FL) commonly involves clients with diverse communication and computational capabilities.
arXiv:2610.07859v1 Announce Type: new Abstract: Conventional decentralized federated learning (DFL) often focuses on clients, with each client maintaining a model copy, performing updates individuall...
arXiv:2608. 09208v1 Announce Type: cross Abstract: Decentralized intelligence systems with heterogeneous devices and limited coordination increasingly rely on decentralized federated learning (DFL).
We study group distributionally robust optimization (GDRO) over decentralized networks, where workers collaborate to learn robust models by minimizing the maximum local risk. Existing methods optimize...
arXiv:2610. 29261v1 Announce Type: cross Abstract: We study group distributionally robust optimization (GDRO) over decentralized networks, where workers collaborate to learn robust models by minimizing the maximum local risk.
arXiv:2603. 01730v2 Announce Type: replace Abstract: Decentralized federated learning (DFL) has emerged as a transformative server-free paradigm that enables collaborative learning over large-scale heterogeneous networks.
SeedFlood is a novel decentralized fine‑tuning method for large language models that scales to billions of parameters and hundreds of clients. It leverages the seed‑reconstructible structure of zeroth‑order gradients to reduce message sizes to near‑zero, enabling efficient flooding across the network. Experiments show SeedFlood outperforms standard zeroth‑order baselines in communication efficiency and generalization, and rivals first‑order gossip methods while incurring far less communication cost.
Conventional decentralized federated learning (DFL) often focuses on clients, with each client maintaining a model copy, performing updates individually, and undertaking model exchange and integration...
Decentralized intelligence systems with heterogeneous devices and limited coordination increasingly rely on decentralized federated learning (DFL). However, DFL suffers from convergence inefficiency under data heterogeneity due to the use of a uniform learning rate (LR) that ignores layer-specific optimization needs.