arXiv:2402. 09589v2 Announce Type: replace-cross Abstract: We present MLCC, a novel technique to augment today's congestion control algorithms to accelerate DNN training jobs in shared GPU clusters in a fully distributed manner.
arXiv:2609.36070v1 Announce Type: cross Abstract: AI accelerator systems are rapidly consolidating into scale-up architectures, where tens to thousands of GPUs communicate over high-bandwidth, single...
arXiv:2607. 28633v1 Announce Type: cross Abstract: Disaggregated LLM inference creates a datacenter networking problem that no existing system solves correctly.
The paper reports an empirical scalability study of data‑parallel training for Kolmogorov‑Arnold Networks (KANs) on high‑performance computing systems. Using up to eight NVIDIA A100 GPUs across four nodes on the FinisTerrae III supercomputer, the authors evaluate strong and weak scaling, communication overhead, and model‑size scaling, finding a 74.7% parallel efficiency and a 5.97× speedup at eight GPUs. They observe non‑monotonic communication costs driven by All‑Reduce choices and inter‑node latency, and note that while the parameter‑to‑memory ratio improves with larger models, training time scales less favorably, leading to guidelines for GPU topology and model‑size selection.
arXiv:2606. 10440v1 Announce Type: cross Abstract: Distributed machine learning (ML) is a key paradigm for today's large-scale artificial intelligence applications.
arXiv:2608. 07733v1 Announce Type: cross Abstract: Graph Neural Networks (GNNs) are widely used across domains such as natural sciences, social network analysis, chip design, and recommendation systems.
The paper proposes using the network itself to aid distributed training over a wide area network (WAN). It suggests employing multicast for outbound traffic and in‑line FPGAs for inbound traffic to reduce bottlenecks, extending techniques commonly used in data centers to the WAN. An optimization framework generates synchronization schedules—rotating cliques of compute islands—tailored to the network topology, and demonstrates these ideas on a nine‑city WAN model, showing how schedules adapt to network capabilities.
arXiv:2508.19073v4 Announce Type: replace-cross Abstract: Deep learning training commonly runs on shared multi-tenant GPU servers, where exclusive allocation provides isolation but can leave resource...
arXiv:2607. 23115v1 Announce Type: cross Abstract: This paper aims to enable computation- and communication-efficient GPU sharing across devices within local area networks (LANs), facilitating ubiquitous AI inference on heterogeneous personal devices.
arXiv:2410.22229v2 Announce Type: replace-cross Abstract: Offloading stateful network functions to multi-threaded SoC SmartNICs promises significant performance and cost benefits. However, realizing...
arXiv:2606. 00946v1 Announce Type: cross Abstract: Efficiently serving large language model (LLM) inference tasks is crucial both for user-perceived latency such as time-to-first-token (TTFT) and for GPU utilization.
The paper introduces a communication‑efficient method for adapting large language models on decentralized GPU meshes. It proposes an asynchronous two‑circuit system that uses fast compressed training with activation masking for pipeline‑parallel transfer and compressed data‑parallel synchronization, while a slower anchor circuit performs occasional unmasked passes. A spectral correction optimizer then denoises the masked gradients using these anchor priors, enabling high compression rates and achieving up to 40× throughput gains over internet‑grade connections while matching dense uncompressed performance.