arXiv:2302. 09832v4 Announce Type: replace Abstract: In distributed optimization and federated learning, slow and costly communication between parallel devices and the central server constitutes the primary bottleneck.
By Laurent Condat, Ivan Agarsk\'y, Grigory Malinovsky, Peter Richt\'arik
arXiv:2605. 01989v2 Announce Type: replace Abstract: Distributed machine learning (ML) training has become a necessity with the prevalence of billion to trillion-parameter-scale models.
By Zechen Ma, Zixi Qu, Jinyan Yi, David Lin, Yashar Ganjali
DART-FL is a multitask federated learning framework designed for edge devices that must balance online inference and model training under limited resources. It dynamically allocates resources between inference and training based on current inference backlog and service capacity, then distributes remaining training capacity among tasks using a queue‑aware scheduler that adjusts loss weights. Experiments on image classification datasets with synthetic and real workloads show that DART‑FL adapts to bursty inference demand, improving accuracy for high‑demand tasks while preserving overall multitask performance.
By Yiming Xie, Pinrui Yu, Geng Yuan, Xue Lin, Ningfang Mi
FedLore introduces a communication- and memory-efficient federated learning framework that shares a low-rank optimization basis across clients each round, mitigating subspace fragmentation and enabling exact low-rank aggregation. By refreshing this shared basis across rounds, FedLore allows model updates to exceed the per-round rank budget while maintaining a provable $O(T^{-1/2})$ stationarity bound under standard assumptions. Experiments on vision and language tasks, including federated pre‑training, demonstrate that FedLore outperforms low‑rank adapter baselines and matches or surpasses full‑parameter training while reducing communication and optimizer‑state memory.
By Junkang Liu
arXiv:2609.36662v1 Announce Type: cross
Abstract: The pre-training of Large Language Models (LLMs) is increasingly conducted across multiple data centers. As training scales to a larger number of acc...
By Pengyu He, Yan Zhang, Ruien Li, Guangwen Yang
The paper introduces Layer-wise Budgeted Adaptive Transmission (LBAT), a method that treats federated learning communication as a resource allocation problem under tight uplink budgets. LBAT estimates the transmission value of each layer using local training signals and then applies a byte‑level dynamic programming allocator to choose optimal rank and bit configurations for each layer. Experiments on heterogeneous federated tabular prediction and data generation tasks show that LBAT outperforms uniform compression baselines, achieving better communication‑utility tradeoffs while maintaining distributional fidelity.
By Pengfei Li, Mohammad Khalil