← Back to all news
arXiv AI By Adam Piaseczny, Md Kamran Chowdhury Shisher, Shiqiang Wang, Christopher G. Brinton

Distributed Learning with Selective State Space Models: Architecture-Aware Convergence Analysis

Read the original on arXiv AI →

The paper investigates how modern selective state space models (SSMs), like Mamba2, behave in distributed learning settings. It derives architecture‑aware gradient and smoothness bounds for single‑ and multi‑layer selective SSMs, and provides convergence bounds for FedAvg and FedProx that incorporate recurrent stability, input‑dependent discretization, and state‑projection norms. Numerical experiments validate the theoretical bounds and compare nine federated learning algorithms on Mamba2 language modeling across six text domains, demonstrating that SSM‑specific bounds help interpret practical federated learning behavior.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Jun 10

FedSLoP: Memory-Efficient Federated Learning with Low-Rank Gradient Projection

arXiv:2604. 24012v3 Announce Type: replace Abstract: Federated learning enables a population of clients to collaboratively train machine learning models without exchanging their raw data, but standard algorithms such as FedAvg suffer from slow convergence and high communication and memory costs in heterogeneous, resource-constrained environments.

By Yutong He, Zhengyang Huang, Jiahe Geng, Kun Yuan
More like this →
arXiv Machine Learning
Aug 27

Differentiated Aggregation to Improve Generalization in Federated Learning

The paper proposes a new federated learning approach called FedALS that reduces communication costs by varying aggregation frequencies across model layers. It derives tighter generalization bounds for one‑round and multi‑round federated learning, linking these bounds to local updates and data heterogeneity. Based on representation‑learning insights, the authors argue that infrequent aggregation of early layers and more frequent aggregation of final layers yields more generalizable models, especially in non‑iid settings, and demonstrate the method’s effectiveness experimentally.

By Peyman Gholami, Hulya Seferoglu
More like this →