arXiv Machine Learning September 24, 2026 By Shengjun Zhang, Tingyi Liu, Heng Zhang, Dong Xie

ZO-COSMO: Index-Free One-Hop Mixing for Decentralized Zeroth-Order Optimization

Read the original on arXiv Machine Learning →

The paper introduces ZO-COSMO, an index‑free one‑hop mixing scheme for decentralized zeroth‑order optimization that couples two‑query estimation with average‑preserving masked consensus. It characterizes the necessary one‑hop condition for sparse communication, derives sharp contraction bounds and convergence guarantees for core and sparse‑momentum updates, and demonstrates empirical gains on synthetic agents and Qwen LoRA workers, achieving notable accuracy improvements over traditional Rand‑k and all‑neighbor mixing methods.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Hugging Face Trending Papers

Sep 23

ZO-COSMO: Index-Free One-Hop Mixing for Decentralized Zeroth-Order Optimization

Sparse communication in decentralized zeroth-order learning requires compatible peer-state coordinates. We characterize this one-hop condition and develop \textsf{ZO-COSMO}, coupling two-query estimat...

agents fine-tuning benchmarks

arXiv Machine Learning

1d ago

SeedFlood: A Step Toward Scalable Decentralized Fine-Tuning of LLMs

SeedFlood is a novel decentralized fine‑tuning method for large language models that scales to billions of parameters and hundreds of clients. It leverages the seed‑reconstructible structure of zeroth‑order gradients to reduce message sizes to near‑zero, enabling efficient flooding across the network. Experiments show SeedFlood outperforms standard zeroth‑order baselines in communication efficiency and generalization, and rivals first‑order gossip methods while incurring far less communication cost.

By Jihun Kim, Dongyeop Lee, Namhoon Lee

llms fine-tuning

arXiv Machine Learning

Aug 11

Decentralized Nonconvex Composite Federated Learning with Gradient Tracking and Momentum

arXiv:2504. 12742v2 Announce Type: replace Abstract: Decentralized Federated Learning (DFL) enables collaborative model training without relying on a central server.

By Yuan Zhou, Xinli Shi, Xuelong Li, Jiachen Zhong, Guanghui Wen, Jinde Cao

benchmarks

arXiv Machine Learning

Jul 27

Variance-Reduced Q-Learning over Static and Time-Varying Networks

arXiv:2607. 21876v1 Announce Type: new Abstract: We investigate a decentralized reinforcement learning problem involving multiple agents that interact with the same Markov Decision Process (MDP).

By Sreejeet Maity, Feng Zhu, Aritra Mitra, Robert W. Heath Jr

agents reinforcement-learning

arXiv AI

Aug 18

Decentralized Federated Learning for Heterogeneous Multi-Task Semantic Communication

arXiv:2608. 15256v1 Announce Type: new Abstract: Collaborative training in distributed semantic communication (DSC) networks typically relies on decentralized federated learning (DFL).

By Lin Yin, Tiejun Lv, Weicai Li, Xi Yu, Xiaoyu He

safety

arXiv AI

2d ago

FedMIX-P: Mixing Local and Global Preconditioners for Federated Vision and Language Model Training

arXiv:2610.01515v1 Announce Type: cross Abstract: Adaptive preconditioners accelerate model training, but heterogeneous client geometries can bias federated updates even when gradients are evaluated...

By Junkang Liu

llms safety

One email a morning, machine-written

Related stories

ZO-COSMO: Index-Free One-Hop Mixing for Decentralized Zeroth-Order Optimization

SeedFlood: A Step Toward Scalable Decentralized Fine-Tuning of LLMs

Decentralized Nonconvex Composite Federated Learning with Gradient Tracking and Momentum

Variance-Reduced Q-Learning over Static and Time-Varying Networks

Decentralized Federated Learning for Heterogeneous Multi-Task Semantic Communication

FedMIX-P: Mixing Local and Global Preconditioners for Federated Vision and Language Model Training