The paper systematically studies decode‑phase energy consumption of open‑source large language models using different attention architectures—Multi‑Head Attention (MHA), Grouped Query Attention (GQA), and GQA with Sliding Window Attention (SWA). It evaluates four models across varying context lengths, batch sizes, and generation workloads, measuring GPU energy via NVIDIA counters. Findings show that the attention mechanism is the main driver of how energy scales with context length, with MHA models growing steeply, GQA models growing less, and GQA+SWA remaining nearly constant; model size mainly sets absolute energy use, while batching can cut energy per token and latency by up to 87%.
By Molka Chkir, Syed Muhammad Danish, Jos H\"oll, Arghavan Asad
arXiv:2609.07966v1 Announce Type: new
Abstract: Key--value (KV) cache compression is an effective way to reduce the memory overhead of large language model (LLM) inference, particularly for long-cont...
By Michael Wang, Keith Li, Roozbeh Bostandoost
arXiv:2608. 06723v1 Announce Type: cross Abstract: The rapid scaling of Large Language Models (LLMs) has significantly increased computational cost, energy consumption, and inference latency, making accurate estimation essential for sustainable artificial intelligence deployment and hardware-aware design.
By Saeid Shokoufa, Mohammad Erfan Sadeghi, Mehdi Kamal, Massoud Pedram
arXiv:2607. 20806v1 Announce Type: new Abstract: Lightweight large language models (LLMs) are increasingly being deployed locally on personal computers and are expected to play a growing role in resource-constrained edge and mobile environments.
By Tomohiro Harada, Enrique Alba, Gabriel Luque
arXiv:2604. 26968v2 Announce Type: replace-cross Abstract: Key-value (KV) cache memory management is the primary bottleneck limiting throughput and cost-efficiency in large-scale GPU inference serving.
By Sanjeev Rao Ganjihal
arXiv:2606. 02964v1 Announce Type: cross Abstract: Large Language Model (LLM) inference relies on key-value (KV) caches to avoid redundant attention computation.
By Chunan Shi, Yilei Chen, Yilin Chen, Xupeng Miao, Bin Cui
arXiv:2606. 07362v1 Announce Type: new Abstract: As scalable inference services become popular, the cold start latency of an inference engine becomes important.
By Huzaifa Shaaban Kabakibo, Animesh Trivedi, Lin Wang
arXiv:2607. 09385v1 Announce Type: cross Abstract: The growing adoption of large language model-based agents within operating system workflows has increased the importance of energy-efficient inference on laptop-class systems-on-chip (SoCs).
By Victor J. B. Jung, Gagandeep Singh, Joseph Melber, Kristof Denolf, Francesco Conti, Luca Benini
arXiv:2609.08307v1 Announce Type: cross
Abstract: Large language models (LLMs) are increasingly used as backends for intelligent web services, but serving them across the edge continuum requires bala...
By Maysam Khatib, Moysis Symeonides, Demetris Trihinas, George Pallis, Marios D. Dikaiakos
arXiv:2607. 13095v1 Announce Type: cross Abstract: We present a full-pipeline inference optimization for the MiMo-V2.
By Xiaomi MiMo Team, Anqi Liu, Aoxin Ma, Bo Chen, Bo Yang, Chen Wang, Chen Zhang, Chengda Tang, Chengwei Wang, Chiheng Lou, Depeng Yan, Fuli Luo, Gang Wang, Hailin Zhang, Jiale Sun, Kang Zhou, Rui Huang, Shaohui Liu, Shen Huang, Shijie Cao, Shuaishuai Fan, Tianling Zhou, Xiangwei Deng, Xueyang Xie, Xuli Wang, Yingchun Lai, Yu Yang, Yuan Zhang, Zhen Tang, Zhonghua Deng, Zihan Jiang
GrowPage is an on‑demand key–value (KV) budgeting framework designed to improve large language model (LLM) reasoning serving. It treats KV capacity as a runtime resource, using lightweight dual‑timescale query summaries to track recent and long‑term attention patterns and estimate demand evolution. At each capacity boundary, GrowPage either compresses KV states within the current allocation or acquires an additional physical page, integrating with PagedAttention’s page‑level memory abstraction to maintain continuous batching and prefix caching.
By Qiankun Ma, Yanjiang Zhou, Zinan Xiong, Haofei Wang, Zhen Song, Yang Xiang, Ziyao Zhang, Hairong Zheng
arXiv:2608.30963v1 Announce Type: cross
Abstract: Modern large language model (LLM) serving systems increasingly operate over repeated or shared context, yet each model typically performs its own pre...
By Yi Li, Dongming Jiang, Yi Zhao, Bingzhe Li