arXiv Machine Learning By Nicol\`o Michelusi

Decentralized Gradient Descent: Bottleneck Regimes and Budget Complexity

Read the original on arXiv Machine Learning →

arXiv:2607. 12172v1 Announce Type: cross Abstract: Decentralized gradient descent (DGD) is widely used for solving distributed optimization problems over networks of agents.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning
1d ago

SeedFlood: A Step Toward Scalable Decentralized Fine-Tuning of LLMs

SeedFlood is a novel decentralized fine‑tuning method for large language models that scales to billions of parameters and hundreds of clients. It leverages the seed‑reconstructible structure of zeroth‑order gradients to reduce message sizes to near‑zero, enabling efficient flooding across the network. Experiments show SeedFlood outperforms standard zeroth‑order baselines in communication efficiency and generalization, and rivals first‑order gossip methods while incurring far less communication cost.

By Jihun Kim, Dongyeop Lee, Namhoon Lee