arXiv AI September 28, 2026 By Philipp M. Z\"ahl, Elja Dalipaj, Anika Hennig, Timon Bayer

Energy Efficiency of Locally Deployed LLMs: A Preliminary Quantitative GPU Power Benchmark on Consumer Hardware

The paper reports a reproducible GPU power benchmark for 18 open‑source LLMs (0.5B–7B parameters) run on a single consumer RTX 4060ti GPU using the Ollama inference engine. Energy metrics such as mean/peak power, total energy per prompt, energy per output token, and throughput were measured, revealing that model architecture and quantization strategy, rather than parameter count alone, drive energy efficiency. The most efficient models were qwen2.5:0.5b and tinyllama:1.1b, while the 7B‑Mistral model consumed up to 8.6× more energy per token, and qwen3.5:0.8b(on) showed unusually high per‑prompt energy due to extended internal reasoning.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning

Aug 31

Characterization of Request and Token Energy Costs for LLM Inference Workloads on GPU Platforms

arXiv:2608.28044v1 Announce Type: cross Abstract: Large language model (LLM) inference serving is priced by tokens, but GPU energy is consumed over inference windows. This accounting mismatch makes t...

By Prabhu Vellaisamy, Vanessa Lam, Shawn Blanton, John Paul Shen

llms

arXiv Machine Learning

Jun 9

LLM Inference at the Edge: Mobile, NPU, and GPU Performance Efficiency Trade-offs Under Sustained Load

arXiv:2603. 23640v2 Announce Type: replace-cross Abstract: Deploying large language models on-device for always-on personal agents demands sustained inference from hardware tightly constrained in power, thermal envelope, and memory.

By Pranay Tummalapalli, Sahil Arayakandy, Ritam Pal, Kautuk Kundan

llms agents efficiency benchmarks

Hugging Face Trending Papers

Jul 2

WattGPU: Predicting Inference Power and Latency on Unseen GPUs and LLMs

Large Language Model (LLM) inference workloads are a rapidly growing contributor to data center energy consumption. Optimizing these deployments requires matching specific LLMs to the most efficient GPUs, but operators currently lack the tools to do so without exhaustively profiling each combination.

llms

arXiv AI

Sep 1

GreenBench: Benchmarking Energy Efficiency and Carbon Footprint of Open-Source LLM Inference on Apple Silicon

arXiv:2608.28667v1 Announce Type: cross Abstract: The rapid proliferation of Large Language Models (LLMs) has raised concerns about their environmental impact during inference. While Green AI researc...

By Rajeswari Kannan, Raj Firke, Shreya Bengle, Srushti Deshmukh

llms nlp efficiency benchmarks

arXiv Machine Learning

Jul 3

WattGPU: Predicting Inference Power and Latency on Unseen GPUs and LLMs

arXiv:2607. 02391v1 Announce Type: cross Abstract: Large Language Model (LLM) inference workloads are a rapidly growing contributor to data center energy consumption.

By Mauricio Fadel Argerich, Jonathan F\"urst, Marta Pati\~no-Mart\'inez

llms

Hugging Face Trending Papers

Jul 29

From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs

The operational energy consumption of large language model (LLM) inference is becoming an increasingly important component of the environmental footprint of deployed AI systems. However, direct measurement of inference energy often requires hardware telemetry, power instrumentation, or infrastructure-specific monitoring, limiting its applicability in comparative studies, early-stage system design, and sustainability reporting.

llms

One email a morning, machine-written

Related stories

Characterization of Request and Token Energy Costs for LLM Inference Workloads on GPU Platforms

LLM Inference at the Edge: Mobile, NPU, and GPU Performance Efficiency Trade-offs Under Sustained Load

WattGPU: Predicting Inference Power and Latency on Unseen GPUs and LLMs

GreenBench: Benchmarking Energy Efficiency and Carbon Footprint of Open-Source LLM Inference on Apple Silicon

WattGPU: Predicting Inference Power and Latency on Unseen GPUs and LLMs

From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs