arXiv Machine Learning
A principled approach for energy-efficient training via phase-aware GPU frequency tuning
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Oct 26
Power- and Fragmentation-aware Online Scheduling for GPU Datacenters
The paper introduces PWR, a scheduling policy that selects power‑efficient GPU and CPU combinations to reduce power consumption in GPU datacenters. It complements the existing Fragmentation Gradient Descent (FGD) policy, which mitigates GPU fragmentation by using a fragmentation metric. Experiments in a simulated cluster show that combining PWR with FGD balances power savings and fragmentation reduction.
arXiv Machine Learning
Sep 14
One Simple Trick for Improving the Performance of Energy-Limited Local Inference and Training
The paper proposes a simple technique of chunking workloads into smaller parts that alternate between compute-intensive and memory-bound operations to smooth power and temperature spikes in GPU systems. By doing so, it prevents throttling, leading to faster wall-clock times and lower total energy consumption. Experiments on a DGX Spark show up to 2% performance and energy gains, while similar benefits, though smaller, are observed on multi‑GPU servers.
arXiv Machine Learning
Aug 24
DAOP: Data-Aware Offloading and Predictive Pre-Calculation for Efficient MoE Inference
arXiv:2501.10375v3 Announce Type: replace-cross Abstract: Mixture-of-Experts (MoE) models, though highly effective for various machine learning tasks, face significant deployment challenges on memory...
arXiv AI
Jul 1
An Efficient Heterogeneous Co-Design for Fine-Tuning on a Single GPU
arXiv:2603. 16428v2 Announce Type: replace-cross Abstract: Fine-tuning Large Language Models (LLMs) has become essential for domain adaptation, but its memory-intensive property exceeds the capabilities of most GPUs.
arXiv AI
Oct 31
The Illusion of Power Capping in LLM Decode: A Phase-Aware Energy Characterisation Across Attention Architectures
The paper investigates the effectiveness of GPU power capping during large language model (LLM) autoregressive decoding. It finds that power caps have minimal impact on decoding performance because decode kernels are memory‑bound and the GPU idles between launches, drawing less than half its 700 W rating. By instead locking the SM clock, the authors can reduce power consumption by up to 32 % with negligible throughput loss and identify three architecture‑dependent frequency‑scaling classes that influence energy use.
arXiv AI
Jun 17
Prefill/Decode-Aware Evaluation of LLM Inference on Emerging AI Accelerators
arXiv:2606. 17104v1 Announce Type: cross Abstract: As large language models (LLMs) are increasingly deployed in latency- and cost-sensitive settings, inference efficiency has become a central systems challenge.