arXiv:2601. 06649v2 Announce Type: replace-cross Abstract: Research in machine learning has questioned whether increases in training token counts reliably produce proportional performance gains in large language models.
arXiv:2606. 13370v1 Announce Type: new Abstract: This study examines training dynamics in a small Llama-style language model trained under a fixed, compute-constrained token budget.
The paper systematically studies decode‑phase energy consumption of open‑source large language models using different attention architectures—Multi‑Head Attention (MHA), Grouped Query Attention (GQA), and GQA with Sliding Window Attention (SWA). It evaluates four models across varying context lengths, batch sizes, and generation workloads, measuring GPU energy via NVIDIA counters. Findings show that the attention mechanism is the main driver of how energy scales with context length, with MHA models growing steeply, GQA models growing less, and GQA+SWA remaining nearly constant; model size mainly sets absolute energy use, while batching can cut energy per token and latency by up to 87%.
The paper investigates the energy costs of multilingual large language model (LLM) inference, revealing significant disparities across languages. Using the ML.Energy framework, the authors find that energy consumption per output token can differ by up to 8.3×, and total energy for a fixed request set can vary up to 179×, with English being the cheapest and Pashto the most expensive. The study attributes these differences to higher per-token costs for complex or rare scripts and longer outputs for low‑resource languages, and notes that high‑energy languages also tend to have lower task accuracy.
arXiv:2608.28044v1 Announce Type: cross Abstract: Large language model (LLM) inference serving is priced by tokens, but GPU energy is consumed over inference windows. This accounting mismatch makes t...
The paper reports a reproducible GPU power benchmark for 18 open‑source LLMs (0.5B–7B parameters) run on a single consumer RTX 4060ti GPU using the Ollama inference engine. Energy metrics such as mean/peak power, total energy per prompt, energy per output token, and throughput were measured, revealing that model architecture and quantization strategy, rather than parameter count alone, drive energy efficiency. The most efficient models were qwen2.5:0.5b and tinyllama:1.1b, while the 7B‑Mistral model consumed up to 8.6× more energy per token, and qwen3.5:0.8b(on) showed unusually high per‑prompt energy due to extended internal reasoning.
arXiv:2511. 20849v2 Announce Type: replace-cross Abstract: We introduce a new tokenizer for language models that minimizes the average tokens per character, thereby reducing the number of tokens needed to represent text during training and to generate text during inference.
arXiv:2507. 07247v2 Announce Type: replace-cross Abstract: As large language models (LLMs) and visual language models (VLMs) grow in scale and application, attention mechanisms have become a central computational bottleneck due to their high memory and time complexity.
arXiv:2607. 26571v1 Announce Type: new Abstract: The operational energy consumption of large language model (LLM) inference is becoming an increasingly important component of the environmental footprint of deployed AI systems.
The operational energy consumption of large language model (LLM) inference is becoming an increasingly important component of the environmental footprint of deployed AI systems. However, direct measurement of inference energy often requires hardware telemetry, power instrumentation, or infrastructure-specific monitoring, limiting its applicability in comparative studies, early-stage system design, and sustainability reporting.
The paper introduces the Power Flexibility Index (PFI) to measure how large language model (LLM) training performance changes when GPU power is reduced. Using 131 training runs on H200 and H100 GPUs, the study finds that LLM jobs have significant but variable power elasticity and identifies telemetry signals that can predict PFI during runtime. The authors demonstrate that allocating power based on PFI maximizes overall token throughput, recovering about 1.5k tokens/s per job under a 30% power reduction, which represents 63% of the gap between equal-weight and perfect-information allocations.
The paper investigates how prompt design influences energy consumption in on-device large language models (LLMs). It examines two prompt properties—cognitive load and phrasing pattern—across various datasets, models, and devices, using phase-level profiling to separate prefill and decode energy. Findings show that cognitive load mainly affects energy per token, while phrasing pattern influences energy mainly through token usage, and that prompt design reshapes the energy-quality trade‑off differently for each model.
arXiv:2607. 25271v1 Announce Type: cross Abstract: Classical compute-optimal scaling laws assume an unbounded supply of fresh pretraining data, yet pretraining is increasingly entering a regime in which compute grows faster than the availability of high-quality data.