LLM-PowerTrace is an open dataset that provides time‑resolved, per‑GPU power traces for large language model fine‑tuning and inference. The traces are sampled every 100 ms across various NVIDIA platforms and include detailed workload and system metadata. The dataset covers a wide range of model families, sizes, and controlled sweeps over batch size, sequence length, precision, parallelism, checkpointing, concurrency, GPU frequency, KV‑cache and weight quantization, prompt and output lengths, and request‑arrival patterns, and it defines power‑grid‑facing metrics and an AI‑compute power & energy scaling law.
arXiv:2607. 26571v1 Announce Type: new Abstract: The operational energy consumption of large language model (LLM) inference is becoming an increasingly important component of the environmental footprint of deployed AI systems.
The operational energy consumption of large language model (LLM) inference is becoming an increasingly important component of the environmental footprint of deployed AI systems. However, direct measurement of inference energy often requires hardware telemetry, power instrumentation, or infrastructure-specific monitoring, limiting its applicability in comparative studies, early-stage system design, and sustainability reporting.
Large Language Model (LLM) inference workloads are a rapidly growing contributor to data center energy consumption. Optimizing these deployments requires matching specific LLMs to the most efficient GPUs, but operators currently lack the tools to do so without exhaustively profiling each combination.
arXiv:2607. 02391v1 Announce Type: cross Abstract: Large Language Model (LLM) inference workloads are a rapidly growing contributor to data center energy consumption.
arXiv:2608.28044v1 Announce Type: cross Abstract: Large language model (LLM) inference serving is priced by tokens, but GPU energy is consumed over inference windows. This accounting mismatch makes t...
The paper reports a reproducible GPU power benchmark for 18 open‑source LLMs (0.5B–7B parameters) run on a single consumer RTX 4060ti GPU using the Ollama inference engine. Energy metrics such as mean/peak power, total energy per prompt, energy per output token, and throughput were measured, revealing that model architecture and quantization strategy, rather than parameter count alone, drive energy efficiency. The most efficient models were qwen2.5:0.5b and tinyllama:1.1b, while the 7B‑Mistral model consumed up to 8.6× more energy per token, and qwen3.5:0.8b(on) showed unusually high per‑prompt energy due to extended internal reasoning.
arXiv:2605. 27599v2 Announce Type: replace-cross Abstract: Agentic AI workloads - where a single user goal triggers multi-step orchestration, tool calls, retries, and failure recovery - are being targeted for edge deployment, with NVIDIA, Dell, HP, ASUS, MSI, Acer, and Gigabyte all shipping GB10-based desktop AI systems in 2026.
arXiv:2609.09662v1 Announce Type: cross Abstract: Deploying Large Language Models (LLMs) directly on mobile platforms at the edge is gaining traction due to a myriad of benefits, such as increased pr...
arXiv:2607. 05475v1 Announce Type: cross Abstract: Deploying Large Language Models (LLMs) on mobile devices enhances privacy and reduces latency, but is severely bottlenecked by hardware inefficiency.
arXiv:2610.14389v1 Announce Type: new Abstract: Benchmark scores tell us what an LLM can do, but not how much energy is required to deliver that capability. We directly measure this trade-off for 86...
The paper introduces the Power Flexibility Index (PFI) to measure how large language model (LLM) training performance changes when GPU power is reduced. Using 131 training runs on H200 and H100 GPUs, the study finds that LLM jobs have significant but variable power elasticity and identifies telemetry signals that can predict PFI during runtime. The authors demonstrate that allocating power based on PFI maximizes overall token throughput, recovering about 1.5k tokens/s per job under a 30% power reduction, which represents 63% of the gap between equal-weight and perfect-information allocations.
arXiv:2610.17531v1 Announce Type: cross Abstract: While it is known that quantization of large language models (LLMs) reduces memory usage via lower-bit weights, studies quantifying the resulting imp...