arXiv Machine Learning
SCOPE: Scalable Training-Free Semantic Coreset Selection for Long-Tailed Federated Learning
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Sep 18
Accelerating Sharded Data Parallelism at Scale with Federated Learning
The paper proposes two hybrid algorithms, FL+FSDP and FL+HSDP, that combine sharded data parallelism with federated learning-style aggregations to reduce communication overhead in large-scale AI training. By partitioning GPUs into loosely‑coupled federation groups, the methods keep inter‑group traffic minimal while maintaining a bounded global batch size. Experiments on a Llama3.1 8B model trained on 512 A100 GPUs show up to 8.04× faster data processing and 4.48 lower evaluation perplexity compared to conventional sharded DP.
arXiv Machine Learning
Aug 11
FEAST: Federated Shared-Space Training for Resource-Heterogeneous Clients
arXiv:2608. 09250v1 Announce Type: new Abstract: Federated learning (FL) must serve devices with varying computational capabilities.
arXiv Machine Learning
Jul 21
Taurus: Accelerating Out-of-Core Graph Neural Network Inference on Billion-Scale Graphs
arXiv:2607. 17374v1 Announce Type: cross Abstract: Graph Neural Network (GNN) inference on billion-scale graphs is challenging due to the large memory footprint of features and embeddings and high disk I/O costs in out-of-core settings.
arXiv AI
Jun 24
Decentralised AI Training and Inference with BlockTrain
arXiv:2606. 24722v1 Announce Type: new Abstract: Frontier AI training is increasingly shaped by access to dense, centrally controlled accelerator clusters.
arXiv AI
Jun 12
MiniMax Sparse Attention
arXiv:2606. 13392v1 Announce Type: new Abstract: Ultra-long-context capability is becoming indispensable for frontier LLMs: agentic workflows, repository-scale code reasoning, and persistent memory all require the model to jointly attend over hundreds of thousands to millions of tokens, yet the quadratic cost of softmax attention makes this untenable at deployment scale.
Hugging Face Trending Papers
Sep 17
Accelerating Sharded Data Parallelism at Scale with Federated Learning
The paper proposes two hybrid algorithms, FL+FSDP and FL+HSDP, that combine sharded data parallelism with federated learning-style aggregations to reduce communication overhead in large-scale AI training. By forming loosely‑coupled federation groups, the methods keep inter‑group traffic minimal while maintaining a bounded global batch size. Experiments on a Llama3.1 8B model trained on 512 A100 GPUs show up to 8.04× faster data processing and 4.48 lower evaluation perplexity compared to traditional sharded DP approaches.