The paper investigates how modern selective state space models (SSMs), like Mamba2, behave in distributed learning settings. It derives architecture‑aware gradient and smoothness bounds for single‑ and multi‑layer selective SSMs, and provides convergence bounds for FedAvg and FedProx that incorporate recurrent stability, input‑dependent discretization, and state‑projection norms. Numerical experiments validate the theoretical bounds and compare nine federated learning algorithms on Mamba2 language modeling across six text domains, demonstrating that SSM‑specific bounds help interpret practical federated learning behavior.
arXiv:2604. 24012v3 Announce Type: replace Abstract: Federated learning enables a population of clients to collaboratively train machine learning models without exchanging their raw data, but standard algorithms such as FedAvg suffer from slow convergence and high communication and memory costs in heterogeneous, resource-constrained environments.
The paper proposes a new federated learning approach called FedALS that reduces communication costs by varying aggregation frequencies across model layers. It derives tighter generalization bounds for one‑round and multi‑round federated learning, linking these bounds to local updates and data heterogeneity. Based on representation‑learning insights, the authors argue that infrequent aggregation of early layers and more frequent aggregation of final layers yields more generalizable models, especially in non‑iid settings, and demonstrate the method’s effectiveness experimentally.
arXiv:2609.39074v1 Announce Type: cross Abstract: Federated learning (FL) on memory-constrained edge devices faces a dilemma: first-order (FO) optimization (i.e., backpropagation) demands substantial...
arXiv:2502. 10239v3 Announce Type: replace-cross Abstract: Federated Learning (FL) is a promising paradigm for finetuning Large Language Models (LLMs) across distributed data sources while preserving data privacy.
arXiv:2511. 19959v2 Announce Type: replace Abstract: Federated learning (FL) has been extensively studied as a privacy-preserving training paradigm.
arXiv:2606. 28835v1 Announce Type: cross Abstract: Federated Learning (FL) emerged as a promising distributed machine learning paradigm.
\textit{Split Federated Learning} (SFL) enables distributed model training by splitting networks between the server and clients. However, under client heterogeneity, the conventional static split stra...
arXiv:2608. 15639v1 Announce Type: cross Abstract: \textit{Split Federated Learning} (SFL) enables distributed model training by splitting networks between the server and clients.
arXiv:2311.03154v3 Announce Type: replace Abstract: There are two categories of methods in Federated Learning (FL) for joint training across multiple clients: (i) parallel FL (PFL), where clients tra...
arXiv:2602. 04879v3 Announce Type: replace-cross Abstract: Reinforcement learning (RL) has become a cornerstone for fine-tuning Large Language Models (LLMs), with Proximal Policy Optimization (PPO) serving as the de facto standard algorithm.
arXiv:2607. 04170v1 Announce Type: new Abstract: Federated Learning (FL) enables decentralized training without data sharing, but suffers from statistical heterogeneity across clients, leading to client drift, poor generalization, and sharp minima compared to centralized training.
The paper introduces Batched SGD, a variant that groups online samples into epochs and performs a single update per epoch using a low‑variance gradient estimate. This batching approach allows a straightforward high‑probability analysis without restrictive assumptions or auxiliary sequences, yielding near‑optimal rates for both strongly convex and non‑convex objectives under standard smoothness and sub‑Gaussian noise conditions. The authors also extend the method to federated learning, providing the first high‑probability guarantees with logarithmic communication complexity, linear speedup in the number of agents, and robustness to data heterogeneity.