arXiv Machine Learning

Energy Estimation for Virtual Servers Using Resource Utilization and Machine Learning

The paper presents a machine learning method to estimate the CPU-package power of individual virtual servers using only metrics observable within the server. Training uses host-side RAPL measurements, while inference relies solely on in‑guest data. Experiments across diverse CPUs, workloads, and configurations show strong predictions when training data match deployment conditions, with some limitations in zero‑shot scenarios, and the approach adds minimal overhead.

arXiv AI

Sep 12

Characterizing Job Power Elasticity for Power-Flexible AI Training

The paper introduces the Power Flexibility Index (PFI) to measure how large language model (LLM) training performance changes when GPU power is reduced. Using 131 training runs on H200 and H100 GPUs, the study finds that LLM jobs have significant but variable power elasticity and identifies telemetry signals that can predict PFI during runtime. The authors demonstrate that allocating power based on PFI maximizes overall token throughput, recovering about 1.5k tokens/s per job under a 30% power reduction, which represents 63% of the gap between equal-weight and perfect-information allocations.

By Philip Colangelo, Charles Dawson, Shayan Sengupta, Ayse Coskun, Varun Sivaram

llms fine-tuning efficiency

arXiv Computation and Language

Sep 10

PELM: Power Efficient On-Device LLM Inference with Speculative Decoding and Dynamic Voltage Frequency Scaling

arXiv:2609.09662v1 Announce Type: cross Abstract: Deploying Large Language Models (LLMs) directly on mobile platforms at the edge is gaining traction due to a myriad of benefits, such as increased pr...

By Weisi Yang, Stephen Xia

llms benchmarks safety

arXiv Machine Learning

Sep 14

HoliBench: A Cross-Platform Benchmarking and Deployment Toolkit for Foundation Models in CPS-IoT Applications

HoliBench is a modular benchmarking and deployment toolkit that jointly measures accuracy, latency, and energy for foundation models across a wide range of devices, from single-board computers to GPU servers. It provides a platform abstraction layer that calibrates cross-device measurements and supports multiple model modalities, inference engines, and quantization levels. Using HoliBench, the authors evaluated 20 models on 7 device types, revealing tradeoffs such as limited latency gains from quantization on low‑precision hardware and diminishing accuracy returns relative to energy consumption, while also showing that single-model profiles can predict multi-model pipeline performance within a few percent.

By Inesh Chakrabarti, Zejun Xiong, Pragya Sharma, Mani Srivastava

llms efficiency multimodal benchmarks

arXiv AI

Sep 10

A Measurement Study of LLM Inference Trade-offs Across Edge Continuum Hardware

arXiv:2609.08307v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as backends for intelligent web services, but serving them across the edge continuum requires bala...

By Maysam Khatib, Moysis Symeonides, Demetris Trihinas, George Pallis, Marios D. Dikaiakos

llms efficiency benchmarks

arXiv AI

Jul 28

ML-based Predictive Models for Power Consumption in Virtualised O-RANs

arXiv:2607. 24256v1 Announce Type: cross Abstract: As communication networks adopt virtualized and disaggregated architectures, achieving energy efficiency has become increasingly important for both economic and environmental reasons.

By Rishu Raj, Genevieve Akude, Urooj Tariq, Daniel Kilper

agents

arXiv Machine Learning

Jul 7

Beyond the Need for Speed: Energy-Aware Code Generation via Simulation-Guided Reinforcement Learning

arXiv:2607. 04577v1 Announce Type: new Abstract: Code models strictly prioritize functional correctness, leaving software energy efficiency as an unoptimized byproduct.

By Saurabhsingh Rajput, Tushar Sharma

reinforcement-learning fine-tuning efficiency

arXiv AI

Jul 13

Attention to Detail: Evaluating Energy, Performance, and Accuracy Trade-offs Across vLLM Configurations

arXiv:2607. 09172v1 Announce Type: cross Abstract: Large Language Models are reshaping how software is developed and maintained.

By Nada Zine, Tristan Coignion, Vincenzo Stoico, Cl\'ement Quinton, Romain Rouvoy, Patricia Lago

llms

arXiv AI

Jul 8

Is Your NPU Ready for LLMs? Dissecting the Hidden Efficiency Bottlenecks in Mobile LLM Inference

arXiv:2607. 05475v1 Announce Type: cross Abstract: Deploying Large Language Models (LLMs) on mobile devices enhances privacy and reduces latency, but is severely bottlenecked by hardware inefficiency.

By Guanyu Cai, Ruiming Tian, Lang Yang, Zhouhong Ren, Jinliang Yuan, Lingkun Li, Jiliang Wang

llms efficiency safety

arXiv AI

Jul 22

CWind: A Cross-site Router for Large Language Model Inference Serving at Renewable Energy Farms

arXiv:2605. 23348v2 Announce Type: replace-cross Abstract: AI power demand is growing at an unprecedented rate while power grids are often ailing and struggle to keep up.

By Tella Rajashekhar Reddy, Atharva Deshmukh, Liangcheng Yu, Chaojie Zhang, Mike Shepperd, Rohan Gandhi, Anjaly Parayil, Srinivasan Iyengar, Ajay Manchepalli, Debopam Bhattacherjee

llms efficiency

Hugging Face Trending Papers

Jul 6

Beyond the Need for Speed: Energy-Aware Code Generation via Simulation-Guided Reinforcement Learning

Code models strictly prioritize functional correctness, leaving software energy efficiency as an unoptimized byproduct. Training models to generate energy-efficient code requires reproducible feedback at scale, which physical hardware measurement cannot reliably provide due to variance.

reinforcement-learning fine-tuning efficiency

arXiv Machine Learning

Aug 5

Design-Time Optimization of Deep Neural Networks for Intermittent Learning on Microcontrollers

arXiv:2608. 03589v1 Announce Type: new Abstract: We present a method for designing deep neural networks (DNNs) for intermittent, energy-autonomous, on-device learning on microcontroller units (MCUs).

By Jakob Schubert, Maximilian Kasper, Maximilian Linke, Benedict Herzog, Mark Deutel, Axel Plinge, Dominik Seuss, Christopher Mutschler

agents

arXiv Machine Learning

Jun 9

Accelerating Divisible Load Processing Through Machine Learning: A Practical Framework for Large-Scale Workloads

arXiv:2605. 23247v2 Announce Type: replace Abstract: In this paper, we introduce the first machine learning framework for predicting optimal processing times in Single-Level Tree Network (SLTN) architectures for the Divisible Load Theory (DLT) paradigm.

By Bharadwaj Veeravalli

Related stories