arXiv AI

LFHE: Local-First Heuristic Evolution for Bounded Local Topology Search in Decentralized Learning with Non-IID Data

arXiv AI
Oct 9

Population Scaling or Data Dilution? Dynamics of Local Topology Evolution in Decentralized Learning

The paper investigates how scaling the number of clients in decentralized learning affects information propagation, consensus, and accuracy. By comparing a degree‑two Ring, a Static Random graph, and a locally adaptive Local‑First Heuristic Evolution (LFHE) topology on CIFAR‑10, the authors show that the Ring’s spectral gap decays as Θ(N⁻²), leading to slower convergence and higher disagreement. Experiments reveal that keeping local dataset size constant mitigates population penalties, and that LFHE can improve accuracy and consensus at the cost of increased communication.

By Yin-Kuan Liang, Yan Gao, Yang Long
arXiv AI
4d ago

Connectivity-Aware Client Selection in Decentralized Federated Learning

The paper introduces connectivity-aware client selection for decentralized federated learning, using spectral graph tools to choose a subset of structurally relevant neighbors for each client. By leveraging eigenvector embeddings and heat/diffusion kernel similarities, the method reduces communication overhead while maintaining or improving accuracy compared to uniform subsampling and full broadcasting. Experiments show up to a 3‑percentage‑point accuracy gain under strong data heterogeneity and a 50% reduction in transmitted messages.

By Usevalad Milasheuski, Monica Nicoli, Stefano Savazzi
arXiv Machine Learning
Jul 7

Air-Plan: Query-Optimized Topology Selection for Over-the-Air Decentralized Federated Learning

arXiv:2607. 04254v1 Announce Type: cross Abstract: Over-the-air (OTA) aggregation exploits the superposition property of wireless multiple-access channels to aggregate model updates from multiple devices within a single transmission slot, significantly reducing communication latency.

By Kaushal Attaluri, Rebeca P. Diaz-Redondo, Manuel Fernandez Veiga
arXiv AI
Jul 7

Decentralised Federated Learning over Temporal Networks: The Role of Heterogeneities

arXiv:2607. 03171v1 Announce Type: cross Abstract: Decentralised federated learning, based on peer-to-peer communication, is increasingly proposed for on-device training of machine learning models, promising a privacy-preserving, communication-efficient training process with no risk of single-point failure.

By Arash Badie-Modiri, Chiara Boldrini, Lorenzo Valerio, J\'anos Kert\'esz, M\'arton Karsai
arXiv Machine Learning
Sep 3

D-FROST: Decentralized Federated pRompt-tuning via Optimal tranSporT for Non-IID and Imbalanced Data

The paper introduces D-FROST, an optimal‑transport‑based algorithm for decentralized federated prompt‑tuning, addressing challenges of heterogeneous local data and non‑IID distributions. It formulates prompt tuning as a Wasserstein optimization over prompt measures and merges neighborhood prompts into compact representative sets. The authors provide theoretical guarantees on consensus error and convergence, and demonstrate effectiveness through experiments on heterogeneous client data.

By Quan Minh Nguyen, Hoang M. Ngo, Trong Nghia Hoang, My T. Thai