arXiv AI

GRANITE : a Byzantine-Resilient Dynamic Gossip Learning Framework

arXiv:2504. 17471v2 Announce Type: replace-cross Abstract: Gossip Learning (GL) is a decentralized learning paradigm where users iteratively exchange and aggregate models with a small set of neighboring peers.

arXiv AI
Sep 2

Optimizing Byzantine Node Placement in Decentralized Federated Learning

The paper investigates how the strategic placement of Byzantine nodes in decentralized federated learning (DFL) affects the propagation of malicious influence across the communication graph. It introduces Byzantine Placement Influence (BPI), a measure that captures cumulative exposure of honest nodes to Byzantine sources over time, and develops algorithms to optimize BPI across various network structures and attack types. Experiments demonstrate that BPI-guided placements consistently yield highly damaging configurations, highlighting the importance of considering node placement in DFL threat models.

By Edoardo Gabrielli, Gabriele Tolomei
arXiv Machine Learning
Sep 4

A Nesterov-Accelerated Byzantine-Robust Federated Learning

The paper proposes Byrd-NAFL, a Byzantine‑robust federated learning algorithm that incorporates Nesterov’s momentum and resilient aggregation rules. It achieves fast and safe convergence under non‑convex, smooth loss functions with relaxed gradient assumptions, and provides a finite‑time convergence guarantee. Experiments show that Byrd-NAFL outperforms existing methods in convergence speed, accuracy, and resilience to various malicious attacks.

By Lihan Xu, Xiaoyi Fan, Gang Wang, Runhao Zeng, Xiping Hu, Yanjie Dong
arXiv Machine Learning
1d ago

SeedFlood: A Step Toward Scalable Decentralized Fine-Tuning of LLMs

SeedFlood is a novel decentralized fine‑tuning method for large language models that scales to billions of parameters and hundreds of clients. It leverages the seed‑reconstructible structure of zeroth‑order gradients to reduce message sizes to near‑zero, enabling efficient flooding across the network. Experiments show SeedFlood outperforms standard zeroth‑order baselines in communication efficiency and generalization, and rivals first‑order gossip methods while incurring far less communication cost.

By Jihun Kim, Dongyeop Lee, Namhoon Lee
arXiv Machine Learning
Aug 3

Communication-Efficient Secure Aggregation in Decentralized Learning

arXiv:2405. 07708v3 Announce Type: replace Abstract: Decentralized learning (DL) enables participants to collaboratively train models without a central server, yet it faces significant scalability challenges that demand sparsification to reduce the prohibitive communication costs of peer-to-peer exchange.

By Sayan Biswas, Anne-Marie Kermarrec, Rafael Pires, Rishi Sharma, Milos Vujasinovic