Hugging Face Blog
Impactful scheduling for GPU clusters
Read the original on Hugging Face Blog →The Flow has not summarised this story yet — read it at Hugging Face Blog.
Related stories
Hugging Face Trending Papers
Oct 26
Plenty of Room in the Middle: Revealing Opportunities on a GPU Cluster via Fine-Grained Monitoring
GPU clusters have become essential computing resources for AI and scientific research in academia and industry. However, their high cost and growing demand often result in long queueing delays. While...
arXiv AI
Oct 28
Plenty of Room in the Middle: Revealing Opportunities on a GPU Cluster via Fine-Grained Monitoring
The paper presents a fine‑grained monitoring study of a university GPU cluster, collecting data at 100 ms intervals using a lightweight tool built on Nvidia’s DCGM library. Analysis of a 10‑day period with 7,399 jobs shows that over 70 % of jobs use less than 20 % of GPU memory and 50 % have below 5 % SM utilization, while 80 % of jobs experience a large mismatch between requested and actual runtimes. The authors also observe highly periodic communication patterns in multi‑GPU jobs, indicating that significant performance gains could be achieved through improved scheduling and multitasking rather than hardware upgrades.
arXiv AI
Oct 26
Power- and Fragmentation-aware Online Scheduling for GPU Datacenters
The paper introduces PWR, a scheduling policy that selects power‑efficient GPU and CPU combinations to reduce power consumption in GPU datacenters. It complements the existing Fragmentation Gradient Descent (FGD) policy, which mitigates GPU fragmentation by using a fragmentation metric. Experiments in a simulated cluster show that combining PWR with FGD balances power savings and fragmentation reduction.
arXiv AI
Jun 9
Resource-aware Computation-Communication Overlap for multi-GPU ML Workloads
arXiv:2606. 09200v1 Announce Type: cross Abstract: The rapid growth of large-scale machine learning (ML) has made distributed training across multiple GPUs a fundamental component of modern ML systems.
arXiv Machine Learning
Jul 10
Tessera: Unlocking Heterogeneous GPUs through Kernel-Granularity Disaggregation
arXiv:2604. 10180v2 Announce Type: replace-cross Abstract: Disaggregation maps parts of an AI workload to different types of GPUs, offering a path to utilize modern heterogeneous GPU clusters.
arXiv Machine Learning
Aug 18
Global Simulation-Guided Dynamic Operator Scheduling for Efficient Multi-Tenant Model Serving
arXiv:2608. 15762v1 Announce Type: cross Abstract: Container-granularity scheduling leaves abundant short-lived idle slices within containers unexploited.