arXiv Machine Learning September 14, 2026 By Erik Schultheis, Maximilian Kleinegger, Dan Alistarh

One Simple Trick for Improving the Performance of Energy-Limited Local Inference and Training

Read the original on arXiv Machine Learning →

The paper proposes a simple technique of chunking workloads into smaller parts that alternate between compute-intensive and memory-bound operations to smooth power and temperature spikes in GPU systems. By doing so, it prevents throttling, leading to faster wall-clock times and lower total energy consumption. Experiments on a DGX Spark show up to 2% performance and energy gains, while similar benefits, though smaller, are observed on multi‑GPU servers.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jun 9

Resource-aware Computation-Communication Overlap for multi-GPU ML Workloads

arXiv:2606. 09200v1 Announce Type: cross Abstract: The rapid growth of large-scale machine learning (ML) has made distributed training across multiple GPUs a fundamental component of modern ML systems.

By Minyu Cui, Miquel Pericas

efficiency

arXiv Machine Learning

1d ago

A principled approach for energy-efficient training via phase-aware GPU frequency tuning

arXiv:2609.24205v1 Announce Type: cross Abstract: Modern AI model training imposes unprecedented computational demands, making it a key contributor to datacenter energy consumption. Yet a significant...

By Miguel Braga, J\'ulio Pinto, Rahma Nouaji, Olivier Michaud, Bettina Kemme, Oana Balmau, Cl\'audia Brito, Ricardo Macedo

arXiv AI

Jun 24

CrossPool: Efficient Multi-LLM Serving for Cold MoE Models through KV-Cache and Weight Disaggregation

arXiv:2606. 24506v1 Announce Type: cross Abstract: Emerging LLM services increasingly host many sparse MoE models, yet most models receive sparse requests and remain cold.

By Zhuoren Ye, Tianyu Wo, Dinghao Xue, Mingming Zhang, Yuchen Teng, Chunming Hu, Renyu Yang

llms benchmarks

arXiv Computation and Language

Sep 10

PELM: Power Efficient On-Device LLM Inference with Speculative Decoding and Dynamic Voltage Frequency Scaling

arXiv:2609.09662v1 Announce Type: cross Abstract: Deploying Large Language Models (LLMs) directly on mobile platforms at the edge is gaining traction due to a myriad of benefits, such as increased pr...

By Weisi Yang, Stephen Xia

llms benchmarks safety

arXiv AI

Jul 7

Data Driven Optimization of GPU efficiency for Distributed LLM-Adapter Serving

arXiv:2602. 24044v2 Announce Type: replace-cross Abstract: Large Language Model (LLM) adapters enable low-cost model specialization, but introduce complex caching and scheduling challenges in distributed serving systems where hundreds of adapters must be hosted concurrently.

By Ferran Agullo, Joan Oliveras, Chen Wang, Alberto Gutierrez-Torre, Olivier Tardieu, Alaa Youssef, Jordi Torres, Josep Ll. Berral

llms efficiency benchmarks

Hugging Face Trending Papers

Jul 2

WattGPU: Predicting Inference Power and Latency on Unseen GPUs and LLMs

Large Language Model (LLM) inference workloads are a rapidly growing contributor to data center energy consumption. Optimizing these deployments requires matching specific LLMs to the most efficient GPUs, but operators currently lack the tools to do so without exhaustively profiling each combination.

llms

One email a morning, machine-written

Related stories

Resource-aware Computation-Communication Overlap for multi-GPU ML Workloads

A principled approach for energy-efficient training via phase-aware GPU frequency tuning

CrossPool: Efficient Multi-LLM Serving for Cold MoE Models through KV-Cache and Weight Disaggregation

PELM: Power Efficient On-Device LLM Inference with Speculative Decoding and Dynamic Voltage Frequency Scaling

Data Driven Optimization of GPU efficiency for Distributed LLM-Adapter Serving

WattGPU: Predicting Inference Power and Latency on Unseen GPUs and LLMs