arXiv AI

GGUF-Metadata Prediction of Single-Sequence llama.cpp Throughput Across Three Systems

The paper predicts single‑sequence llama.cpp throughput from GGUF metadata using roofline‑shaped predictors with quantization‑specific scale factors. Experiments on 318 measurements across 53 host‑file configurations on two Apple M4 Max systems and an NVIDIA RTX 5080 show that an active‑parameter decode model achieves significantly lower mean absolute percentage errors compared to models that use total parameters. The study also finds that low‑bit model ladders alter runtime ordering and that GGUF structure improves predictions, though fitted efficiencies vary across systems.

arXiv AI
6d ago

The Inference Engineering Pareto Atlas: Which Optimizations Dominate the Cost, Quality, and Latency Frontier?

The paper presents a Pareto atlas of LLM inference optimizations, mapping cost, quality, and latency trade‑offs for Qwen2.5‑7B‑Instruct on L4, A100, and H100 GPUs. Using 54 measured configurations and a calibrated simulator, it identifies 18 of 36 setups on the Pareto frontier, showing that combined methods outperform single ones. Quality tests reveal that AWQ 4bit and FP8 weights offer significant latency reductions while largely preserving accuracy, but naive FP8 KV caching fails to answer any questions correctly.

By Srikanta Datta Tumkur, Jay Iyer, Mehar Simhadri, Sai Pavan Kumar, Sai Kapil Kumar, Ramesh Nampelly
arXiv AI
Aug 20

TokenPowerSandbox: Evidence-Gated CPU-First Screening for Energy-Aware LLM Serving

TokenPowerSandbox is an evidence‑gated workflow that uses a CPU‑resident projector, brief GPU probes, full‑workload verification, and tamper‑evident provenance to predict energy usage of large language model serving. In experiments on an NVIDIA H100 80GB running Qwen2.5‑7B‑Instruct with vLLM, the method achieved energy MAPE of 6.23% and 7.35% on blind holdout and no‑refit confirmations, with high Spearman rank correlations. A predeclared TTFT gate demonstrated that energy accuracy alone cannot guarantee latency, as it passed at concurrency four but abstained below that level.

By Chenxu Niu
arXiv Machine Learning
Sep 14

Dissecting GPU Utilization for LLM Inference on Nvidia Hopper

The paper investigates how a single SM utilization metric can misrepresent the true workload of large language model (LLM) inference on Nvidia Hopper GPUs. By profiling vLLM with FlashAttention‑3 and cuBLASLt on an H100 NVL across various phases (cold prefill, warm prefill, and decode) and varying sequence length and batch size, the authors replace the single utilization figure with eight detailed counter‑validated views. These views, tied to specific Nsight Compute counters or formulas, reveal how factors such as fragment fill, occupancy limits, stall signatures, wave quantization, and kernel selection create utilization gaps across four production models and six per‑layer kernel roles.

By Mohammad Siavashi, Gerald Q. Maguire Jr., Dejan Kostic, Marco Chiesa
arXiv AI
Jun 30

KernelSight-LM: A Kernel-Level LLM Inference Simulator

arXiv:2606. 28565v1 Announce Type: cross Abstract: As large language models (LLMs) move into production serving, practitioners must rapidly evaluate inference performance across diverse hardware, models, and serving parameters to meet cost and latency targets.

By Xiteng Yao, Taeho Kim, Hengzhi Pei, Xinle Liu, Kyle Ulrich, Leonard Lausen, Ashish Khetan, Xiang Song, George Karypis, Martin Herbordt
arXiv Machine Learning
Sep 11

Phase-Decoupled, Model-Calibrated Power Control for Disaggregated LLM Serving

The paper introduces a phase‑decoupled, model‑calibrated power controller for disaggregated large‑language‑model (LLM) serving, addressing the mismatch between GPU power settings and the distinct hardware regimes of prefill and decode stages. By calibrating separate power caps for each lane based on measured throughput‑latency cliffs, the authors achieve a 20.4% increase in tokens per joule with only a 3.5% rise in mean end‑to‑end latency on an 8‑node B200 cluster, outperforming NVIDIA’s Max‑Q profile. The approach also demonstrates consistent meeting of ITL‑p99 service‑level objectives across multiple MoE models and yields a 32.3% electricity savings over a sustained three‑day run.

By Jae Gon Kim, Donghoon Yoo, Hanyul Ryu, Sungho Ha, Juyeon Lee, Soojung Ryu
Hugging Face Trending Papers
Sep 10

Phase-Decoupled, Model-Calibrated Power Control for Disaggregated LLM Serving

The paper evaluates NVIDIA’s Max‑Q inference profile on a disaggregated B200 GPU system for large language model (LLM) serving, finding modest gains (+8.6% tokens/J) but increased latency (+5.2%). It proposes a phase‑decoupled, model‑calibrated power controller that sets a latency‑guaranteed SM‑clock window for prefill and a calibrated power cap for decode, achieving a 20.4% tokens/J improvement with only a 3.5% latency increase on an 8‑node Qwen3‑Coder‑480B deployment. The approach outperforms vendor profiles on both energy and latency, and demonstrates significant long‑term electricity savings in MoE‑based serving.