The paper systematically studies decode‑phase energy consumption of open‑source large language models using different attention architectures—Multi‑Head Attention (MHA), Grouped Query Attention (GQA), and GQA with Sliding Window Attention (SWA). It evaluates four models across varying context lengths, batch sizes, and generation workloads, measuring GPU energy via NVIDIA counters. Findings show that the attention mechanism is the main driver of how energy scales with context length, with MHA models growing steeply, GQA models growing less, and GQA+SWA remaining nearly constant; model size mainly sets absolute energy use, while batching can cut energy per token and latency by up to 87%.
arXiv:2507. 07247v2 Announce Type: replace-cross Abstract: As large language models (LLMs) and visual language models (VLMs) grow in scale and application, attention mechanisms have become a central computational bottleneck due to their high memory and time complexity.
arXiv:2608.28044v1 Announce Type: cross Abstract: Large language model (LLM) inference serving is priced by tokens, but GPU energy is consumed over inference windows. This accounting mismatch makes t...
The operational energy consumption of large language model (LLM) inference is becoming an increasingly important component of the environmental footprint of deployed AI systems. However, direct measurement of inference energy often requires hardware telemetry, power instrumentation, or infrastructure-specific monitoring, limiting its applicability in comparative studies, early-stage system design, and sustainability reporting.
arXiv:2607. 26571v1 Announce Type: new Abstract: The operational energy consumption of large language model (LLM) inference is becoming an increasingly important component of the environmental footprint of deployed AI systems.
The paper reports a reproducible GPU power benchmark for 18 open‑source LLMs (0.5B–7B parameters) run on a single consumer RTX 4060ti GPU using the Ollama inference engine. Energy metrics such as mean/peak power, total energy per prompt, energy per output token, and throughput were measured, revealing that model architecture and quantization strategy, rather than parameter count alone, drive energy efficiency. The most efficient models were qwen2.5:0.5b and tinyllama:1.1b, while the 7B‑Mistral model consumed up to 8.6× more energy per token, and qwen3.5:0.8b(on) showed unusually high per‑prompt energy due to extended internal reasoning.
arXiv:2606. 02964v1 Announce Type: cross Abstract: Large Language Model (LLM) inference relies on key-value (KV) caches to avoid redundant attention computation.
arXiv:2607. 09172v1 Announce Type: cross Abstract: Large Language Models are reshaping how software is developed and maintained.
arXiv:2610.28576v1 Announce Type: new Abstract: Energy-efficient deployment of large language models requires understanding how model architecture, hardware, and serving settings shape energy per gen...
arXiv:2608. 06723v1 Announce Type: cross Abstract: The rapid scaling of Large Language Models (LLMs) has significantly increased computational cost, energy consumption, and inference latency, making accurate estimation essential for sustainable artificial intelligence deployment and hardware-aware design.
arXiv:2610.28223v1 Announce Type: cross Abstract: Large language models can solve complex problems across a variety of subjects, such as mathematics and coding. For such complex analytical tasks, mod...
arXiv:2610.17531v1 Announce Type: cross Abstract: While it is known that quantization of large language models (LLMs) reduces memory usage via lower-bit weights, studies quantifying the resulting imp...
arXiv:2607. 05933v1 Announce Type: cross Abstract: Dynamic Voltage Frequency Scaling (DVFS) on resource-constrained embedded GPU platforms is essential for energy-efficient small language model (SLM) fine-tuning, as privacy- and personalization-driven adaptation increasingly requires local execution and involves repeated forward-backward optimization over many mini-batches, making it substantially more time- and energy-intensive than single-pass inference.