The paper introduces ZO-COSMO, an index‑free one‑hop mixing scheme for decentralized zeroth‑order optimization that couples two‑query estimation with average‑preserving masked consensus. It characterizes the necessary one‑hop condition for sparse communication, derives sharp contraction bounds and convergence guarantees for core and sparse‑momentum updates, and demonstrates empirical gains on synthetic agents and Qwen LoRA workers, achieving notable accuracy improvements over traditional Rand‑k and all‑neighbor mixing methods.
By Shengjun Zhang, Tingyi Liu, Heng Zhang, Dong Xie
arXiv:2504. 12742v2 Announce Type: replace Abstract: Decentralized Federated Learning (DFL) enables collaborative model training without relying on a central server.
By Yuan Zhou, Xinli Shi, Xuelong Li, Jiachen Zhong, Guanghui Wen, Jinde Cao
SeedFlood is a novel decentralized fine‑tuning method for large language models that scales to billions of parameters and hundreds of clients. It leverages the seed‑reconstructible structure of zeroth‑order gradients to reduce message sizes to near‑zero, enabling efficient flooding across the network. Experiments show SeedFlood outperforms standard zeroth‑order baselines in communication efficiency and generalization, and rivals first‑order gossip methods while incurring far less communication cost.
By Jihun Kim, Dongyeop Lee, Namhoon Lee
arXiv:2607. 21876v1 Announce Type: new Abstract: We investigate a decentralized reinforcement learning problem involving multiple agents that interact with the same Markov Decision Process (MDP).
By Sreejeet Maity, Feng Zhu, Aritra Mitra, Robert W. Heath Jr
arXiv:2608. 15256v1 Announce Type: new Abstract: Collaborative training in distributed semantic communication (DSC) networks typically relies on decentralized federated learning (DFL).
By Lin Yin, Tiejun Lv, Weicai Li, Xi Yu, Xiaoyu He
arXiv:2607. 01665v1 Announce Type: new Abstract: Decentralized online convex optimization (D-OCO) is a popular framework for distributed applications with streaming data.
By Hao Zhou, Xiaoyu Wang, Chang Yao, Mingli Song, Yuanyu Wan
arXiv:2606. 11081v1 Announce Type: cross Abstract: Communication-efficient pre-training of LLMs is increasingly important as training draws on compute distributed across clusters, data centers, and lower-bandwidth links.
By Pietro Cagnasso, Eugene Belilovsky, Edouard Oyallon
Communication-efficient pre-training of LLMs is increasingly important as training draws on compute distributed across clusters, data centers, and lower-bandwidth links. Many practical methods reduce communication frequency but still rely on synchronous All-Reduce operations that maintain identical model states and tie progress to global collectives.
arXiv:2608. 03852v1 Announce Type: new Abstract: This paper proposes FedCritic-MIMO, a communication-efficient serverless federated multi-agent reinforcement learning framework for AI-native resource control across independently deployable cell-level controllers in open and disaggregated 6G RANs.
By Amin Farajzadeh, Melike Erol-Kantarci
arXiv:2610.01515v1 Announce Type: cross
Abstract: Adaptive preconditioners accelerate model training, but heterogeneous client geometries can bias federated updates even when gradients are evaluated...
By Junkang Liu
arXiv:2409. 19279v2 Announce Type: replace-cross Abstract: Continuous-time models can reveal accelerated structures in distributed optimization, but their rates need not survive direct discretization.
By Kushal Chakrabarti, Mayank Baranwal
arXiv:2606. 01717v1 Announce Type: new Abstract: Instruction tuning aligns large language models, including multimodal ones, with diverse user intents, but scaling to heterogeneous mixtures is hindered by gradient interference and bandwidth-heavy synchronization.
By Minsik Choi, Geewook Kim