← Back to all news
arXiv Machine Learning

Energy and Latency First: How Deployment Constraints Reverse Time-Series Model Selection

The paper examines how deployment constraints such as strict latency deadlines and cumulative energy budgets can overturn traditional accuracy-based model selection for time‑series tasks. Using handover prediction in cellular networks as a case study, the authors evaluate eight architectures, screening for deadline feasibility at the tail and then ranking the feasible models by measured inference energy, including feature‑computation costs. Gradient‑boosted trees on handcrafted temporal features achieve the highest accuracy (F1 = 0.974) and are also among the cheapest to run, whereas a neural architecture that would rank first by accuracy alone (CNN‑1D, F1 = 0.914) consumes over three times more energy per inference under realistic duty‑cycled deployment. whyItMatters":"The study demonstrates that accuracy‑only benchmarking can mislead deployment decisions by favoring models that are infeasible under real‑world latency and energy constraints."

Related stories

arXiv Machine Learning
Sep 3

Network-Aware Forecasting on Wireless Access Points

The paper investigates how enterprise wireless access points (APs) can run predictive machine learning models while sharing CPU and memory with essential networking tasks. It introduces the concept of network‑aware deployability, requiring models to qualify for the AP’s hardware and to be validated under real packet‑service constraints. Benchmarks reveal that models can run 6–19× slower and use up to 22% more memory on an AP compared to a Raspberry Pi 5, and that even similarly sized forecasting models can differ by 19× in latency, leading to significant increases in round‑trip time and throughput degradation under load.

By Niloo Bahadori, Swadhin Pradhan, Peiman Amini
efficiencybenchmarks
More like this →
arXiv Machine Learning
Sep 11

Optimizing AI Inference Across the Deployment Stack

The paper argues that AI deployment performance depends on interactions among compression, compiler transformations, and serving policies rather than just model architecture. It introduces a three‑layer taxonomy—model‑level techniques, compiler transformations, and system policies—and frames deployment as a constrained multi‑objective optimization problem over accuracy, latency, throughput, memory footprint, and energy. The authors propose an evidence protocol for comparable benchmarking and synthesize data from edge and data‑center platforms to show that cross‑layer interactions drive deployment outcomes, concluding with a constraint‑aware selection procedure and open research problems.

By Tejinder Singh, John Pflueger, Jeebak Mitra, Robert Lincourt, Mitchell Markow, Bhavesh A. Patel
llmsefficiencybenchmarks
More like this →
Hugging Face Trending Papers
Aug 11

CARB: A Characterization-Guided Framework for CNN Inference Cost Prediction and Deployment Screening

Accurate pre-deployment estimation of CNN inference cost--energy, latency, and peak memory--is increasingly critical as models are deployed on resource-constrained GPU platforms. Existing approaches rely on FLOPs, latency measurements, or single-device profiling as energy proxies, overlooking the non-linear interactions between architectural design and hardware load.

More like this →
arXiv Machine Learning
Sep 14

The Battery Price of edge AI: A study of the Environmental Impact of LLM Inference on Mobile Devices

The paper investigates the environmental impact of running large language models (LLMs) on mobile devices. It evaluates 18 different LLM configurations on two smartphones and a server, measuring energy per token, latency, accuracy, and battery-cycle consumption. Findings reveal that on-device inference is about three times less energy‑efficient than batched server inference, that energy consumption varies non‑monotonically with quantization bit‑width, and that most models are not on the Pareto front of accuracy and energy efficiency. The study concludes that local AI is not inherently more sustainable than cloud inference, with the majority of environmental impact stemming from device embodied carbon.

By \'Edouard Gu\'egain, Tristan Coignion
llmsdiffusionroboticsefficiencybenchmarkssafety
More like this →
arXiv AI
Aug 28

The Accuracy-Efficiency Paradox Quantifying Net Energy Loss in on-Device Energy Forecasting

The paper titled "The Accuracy-Efficiency Paradox Quantifying Net Energy Loss in on-Device Energy Forecasting" discusses how highly accurate energy forecasting models can paradoxically cause a net energy deficit on edge devices due to inference energy consumption and battery aging. It introduces a Total Cost of Ownership (TCO) framework that unifies inference energy and battery degradation as forms of energy loss, aiming to minimize net energy loss. Experiments show that in thermally sensitive edge environments, the energy saved by more precise, complex models is often offset by the higher operational intensity they require.

By Jaeik Jeong, Tai-Yeon Ku, Wan-Ki Park
More like this →