arXiv Machine Learning August 28, 2026 By Nihar Shah, Ben Blier

Distributed Training using an Intelligent Network

Read the original on arXiv Machine Learning →

The paper proposes using the network itself to aid distributed training over a wide area network (WAN). It suggests employing multicast for outbound traffic and in‑line FPGAs for inbound traffic to reduce bottlenecks, extending techniques commonly used in data centers to the WAN. An optimization framework generates synchronization schedules—rotating cliques of compute islands—tailored to the network topology, and demonstrates these ideas on a nine‑city WAN model, showing how schedules adapt to network capabilities.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Aug 17

MLCC: A Congestion Control Technique to Accelerate ML Training

arXiv:2402. 09589v2 Announce Type: replace-cross Abstract: We present MLCC, a novel technique to augment today's congestion control algorithms to accelerate DNN training jobs in shared GPU clusters in a fully distributed manner.

By Anton A. Zabreyko, Sanjoli Narang, Sudarsanan Rajasekaran, Manya Ghobadi

efficiency

arXiv AI

Jun 10

Unifying Local Communications and Local Updates for LLM Pretraining

arXiv:2606. 11081v1 Announce Type: cross Abstract: Communication-efficient pre-training of LLMs is increasingly important as training draws on compute distributed across clusters, data centers, and lower-bandwidth links.

By Pietro Cagnasso, Eugene Belilovsky, Edouard Oyallon

llms benchmarks

arXiv Machine Learning

Aug 7

ML-for-ML

arXiv:2608. 06046v1 Announce Type: cross Abstract: AI training workloads are growing rapidly, making their time, energy, and infrastructure costs increasingly important.

By Yutong Zhao, Noga H. Rotman, Gianni Antichi, Ran Ben Basat

fine-tuning

arXiv Machine Learning

Sep 7

Tuning Collective Patterns to Alleviate Congestion in Shared AI Clusters

The paper introduces REACT, a system that dynamically tunes communication collectives in distributed AI training to mitigate congestion without requiring network infrastructure changes. REACT operates at the application layer, detecting congestion via flow statistics and adjusting the pattern of data exchange—such as selecting different aggregation nodes in an AllReduce tree—while preserving the semantics of the communication. Evaluations on a shared academic GPU cluster show that REACT improves algorithm bandwidth by 13%–38% under congestion, with simulations indicating potential gains up to 75%.

By Eashan Gupta, Yongzhou Chen, Apoorve Mohan, Pavlos Maniotis, Abdullah Kayi, Radhika Mittal

Hugging Face Trending Papers

Jun 9

Unifying Local Communications and Local Updates for LLM Pretraining

Communication-efficient pre-training of LLMs is increasingly important as training draws on compute distributed across clusters, data centers, and lower-bandwidth links. Many practical methods reduce communication frequency but still rely on synchronous All-Reduce operations that maintain identical model states and tie progress to global collectives.

llms benchmarks

arXiv AI

Aug 7

Multi-Agent Transformer for Queue-Level XR Traffic Scheduling in TSN Networks

arXiv:2608. 05340v1 Announce Type: cross Abstract: Time-Sensitive Networking (TSN) and Mobile Edge Computing (MEC) hold strong potential for enabling ultra-reliable low-latency communication for time-sensitive applications, such as eXtended Reality (XR).

By Marcos Carvalho, Fatih Temiz, Shavbo Salehi, Melike Erol-Kantarci, Daniel F. Macedo

llms agents reinforcement-learning

One email a morning, machine-written

Related stories

MLCC: A Congestion Control Technique to Accelerate ML Training

Unifying Local Communications and Local Updates for LLM Pretraining

ML-for-ML

Tuning Collective Patterns to Alleviate Congestion in Shared AI Clusters

Unifying Local Communications and Local Updates for LLM Pretraining

Multi-Agent Transformer for Queue-Level XR Traffic Scheduling in TSN Networks