← Back to all news
arXiv Machine Learning By Jae Gon Kim, Donghoon Yoo, Hanyul Ryu, Sungho Ha, Juyeon Lee, Soojung Ryu

Phase-Decoupled, Model-Calibrated Power Control for Disaggregated LLM Serving

Read the original on arXiv Machine Learning →

The paper introduces a phase‑decoupled, model‑calibrated power controller for disaggregated large‑language‑model (LLM) serving, addressing the mismatch between GPU power settings and the distinct hardware regimes of prefill and decode stages. By calibrating separate power caps for each lane based on measured throughput‑latency cliffs, the authors achieve a 20.4% increase in tokens per joule with only a 3.5% rise in mean end‑to‑end latency on an 8‑node B200 cluster, outperforming NVIDIA’s Max‑Q profile. The approach also demonstrates consistent meeting of ITL‑p99 service‑level objectives across multiple MoE models and yields a 32.3% electricity savings over a sustained three‑day run.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

Hugging Face Trending Papers
Sep 10

Phase-Decoupled, Model-Calibrated Power Control for Disaggregated LLM Serving

The paper evaluates NVIDIA’s Max‑Q inference profile on a disaggregated B200 GPU system for large language model (LLM) serving, finding modest gains (+8.6% tokens/J) but increased latency (+5.2%). It proposes a phase‑decoupled, model‑calibrated power controller that sets a latency‑guaranteed SM‑clock window for prefill and a calibrated power cap for decode, achieving a 20.4% tokens/J improvement with only a 3.5% latency increase on an 8‑node Qwen3‑Coder‑480B deployment. The approach outperforms vendor profiles on both energy and latency, and demonstrates significant long‑term electricity savings in MoE‑based serving.

llmsagentsefficiency
More like this →
arXiv AI
Oct 31

The Illusion of Power Capping in LLM Decode: A Phase-Aware Energy Characterisation Across Attention Architectures

The paper investigates the effectiveness of GPU power capping during large language model (LLM) autoregressive decoding. It finds that power caps have minimal impact on decoding performance because decode kernels are memory‑bound and the GPU idles between launches, drawing less than half its 700 W rating. By instead locking the SM clock, the authors can reduce power consumption by up to 32 % with negligible throughput loss and identify three architecture‑dependent frequency‑scaling classes that influence energy use.

By Bole Ma, Ayesha Afzal, Jan Eitzinger, Gerhard Wellein
llmsefficiency
More like this →