GPU clusters have become essential computing resources for AI and scientific research in academia and industry. However, their high cost and growing demand often result in long queueing delays. While...
The paper presents a fine‑grained monitoring study of a university GPU cluster, collecting data at 100 ms intervals using a lightweight tool built on Nvidia’s DCGM library. Analysis of a 10‑day period with 7,399 jobs shows that over 70 % of jobs use less than 20 % of GPU memory and 50 % have below 5 % SM utilization, while 80 % of jobs experience a large mismatch between requested and actual runtimes. The authors also observe highly periodic communication patterns in multi‑GPU jobs, indicating that significant performance gains could be achieved through improved scheduling and multitasking rather than hardware upgrades.
The paper introduces PWR, a scheduling policy that selects power‑efficient GPU and CPU combinations to reduce power consumption in GPU datacenters. It complements the existing Fragmentation Gradient Descent (FGD) policy, which mitigates GPU fragmentation by using a fragmentation metric. Experiments in a simulated cluster show that combining PWR with FGD balances power savings and fragmentation reduction.
arXiv:2606. 09200v1 Announce Type: cross Abstract: The rapid growth of large-scale machine learning (ML) has made distributed training across multiple GPUs a fundamental component of modern ML systems.
arXiv:2604. 10180v2 Announce Type: replace-cross Abstract: Disaggregation maps parts of an AI workload to different types of GPUs, offering a path to utilize modern heterogeneous GPU clusters.
arXiv:2608. 15762v1 Announce Type: cross Abstract: Container-granularity scheduling leaves abundant short-lived idle slices within containers unexploited.
arXiv:2608. 12123v1 Announce Type: cross Abstract: LLM-agent services repeatedly execute small deterministic transitions between model and tool calls: route an outcome, update state, and emit the next effect.
arXiv:2609.27040v1 Announce Type: cross Abstract: Multi-GPU servers have become the standard building block of modern data centers, providing aggregated capacity through high-bandwidth interconnects....
arXiv:2602. 24044v2 Announce Type: replace-cross Abstract: Large Language Model (LLM) adapters enable low-cost model specialization, but introduce complex caching and scheduling challenges in distributed serving systems where hundreds of adapters must be hosted concurrently.
arXiv:2607. 07862v1 Announce Type: cross Abstract: The evolution of compute infrastructure has transformed multi-GPU systems into tightly integrated shared-memory structures.
arXiv:2607. 19539v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) architectures increase model capacity without proportionally increasing computation cost and have become a key building block for scaling large language models (LLMs) to trillion-parameter regimes.
The paper proposes a simple technique of chunking workloads into smaller parts that alternate between compute-intensive and memory-bound operations to smooth power and temperature spikes in GPU systems. By doing so, it prevents throttling, leading to faster wall-clock times and lower total energy consumption. Experiments on a DGX Spark show up to 2% performance and energy gains, while similar benefits, though smaller, are observed on multi‑GPU servers.