← Back to all news
arXiv Computation and Language By Wei Hu, Xiaolong Tu, Dawei Chen, Yitao Chen, Kyungtae Han, Haoxin Wang

How Do Prompt Variations Affect Energy Consumption in On-Device LLMs?

Read the original on arXiv Computation and Language →

The paper investigates how prompt design influences energy consumption in on-device large language models (LLMs). It examines two prompt properties—cognitive load and phrasing pattern—across various datasets, models, and devices, using phase-level profiling to separate prefill and decode energy. Findings show that cognitive load mainly affects energy per token, while phrasing pattern influences energy mainly through token usage, and that prompt design reshapes the energy-quality trade‑off differently for each model.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv Machine Learning
Jun 24

EnerInfer: Energy-Aware On-Device LLM Inference

arXiv:2606. 23001v1 Announce Type: cross Abstract: On-device LLM inference is increasingly attractive for privacy-preserving, reliable, and cost-effective deployment, yet its energy and thermal costs remain a critical bottleneck.

By Bohua Zou, Nian Liu, Binqi Sun, Matteo Mascherin, Debayan Roy, Yutao Liu, Yu Peng, Ning Jia, Haibo Chen
llmsefficiencysafety
More like this →
arXiv Machine Learning
Aug 27

Understanding the Energy Scaling of Large Language Model Inference Across Context Lengths and Attention Architectures

The paper systematically studies decode‑phase energy consumption of open‑source large language models using different attention architectures—Multi‑Head Attention (MHA), Grouped Query Attention (GQA), and GQA with Sliding Window Attention (SWA). It evaluates four models across varying context lengths, batch sizes, and generation workloads, measuring GPU energy via NVIDIA counters. Findings show that the attention mechanism is the main driver of how energy scales with context length, with MHA models growing steeply, GQA models growing less, and GQA+SWA remaining nearly constant; model size mainly sets absolute energy use, while batching can cut energy per token and latency by up to 87%.

By Molka Chkir, Syed Muhammad Danish, Jos H\"oll, Arghavan Asad
llms
More like this →