arXiv Machine Learning September 14, 2026 By Erik Schultheis, Maximilian Kleinegger, Dan Alistarh

One Simple Trick for Improving the Performance of Energy-Limited Local Inference and Training

Read the original on arXiv Machine Learning →

The paper proposes a simple technique of chunking workloads into smaller parts that alternate between compute-intensive and memory-bound operations to smooth power and temperature spikes in GPU systems. By doing so, it prevents throttling, leading to faster wall-clock times and lower total energy consumption. Experiments on a DGX Spark show up to 2% performance and energy gains, while similar benefits, though smaller, are observed on multi‑GPU servers.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jun 9

Resource-aware Computation-Communication Overlap for multi-GPU ML Workloads

arXiv:2606. 09200v1 Announce Type: cross Abstract: The rapid growth of large-scale machine learning (ML) has made distributed training across multiple GPUs a fundamental component of modern ML systems.

By Minyu Cui, Miquel Pericas

efficiency

arXiv Machine Learning

Sep 22

A principled approach for energy-efficient training via phase-aware GPU frequency tuning

arXiv:2609.24205v1 Announce Type: cross Abstract: Modern AI model training imposes unprecedented computational demands, making it a key contributor to datacenter energy consumption. Yet a significant...

By Miguel Braga, J\'ulio Pinto, Rahma Nouaji, Olivier Michaud, Bettina Kemme, Oana Balmau, Cl\'audia Brito, Ricardo Macedo

arXiv AI

Oct 21

Local Inference of LLMs on Long Contexts: A Data Systems Perspective

arXiv:2610.28223v1 Announce Type: cross Abstract: Large language models can solve complex problems across a variety of subjects, such as mathematics and coding. For such complex analytical tasks, mod...

By Mattias D\"urrmeier, Alberto Lerner

llms efficiency

arXiv AI

Jun 24

CrossPool: Efficient Multi-LLM Serving for Cold MoE Models through KV-Cache and Weight Disaggregation

arXiv:2606. 24506v1 Announce Type: cross Abstract: Emerging LLM services increasingly host many sparse MoE models, yet most models receive sparse requests and remain cold.

By Zhuoren Ye, Tianyu Wo, Dinghao Xue, Mingming Zhang, Yuchen Teng, Chunming Hu, Renyu Yang

llms benchmarks

arXiv Machine Learning

Oct 28

DualPace: Decoupled Scheduling and Cooperative GPU--CPU Decoding for LLM Serving

DualPace is an LLM serving runtime that decouples scheduling between a GPU engine and an AMX‑enabled CPU engine, allowing the CPU to independently batch and advance decodes while still cooperating with the GPU for dense stages. This design expands key‑value cache capacity into host memory and reduces bursty CPU progress, leading to significant throughput gains—up to 43.2 % over NEO and 90.6 % over vLLM—without increasing latency. The system also maintains low time‑to‑first‑token and supports higher request rates at comparable tail latency.

By YuXuan Lin, KeQiang Wu, Kingsum Chow

llms efficiency benchmarks

arXiv Computation and Language

Sep 10

PELM: Power Efficient On-Device LLM Inference with Speculative Decoding and Dynamic Voltage Frequency Scaling

arXiv:2609.09662v1 Announce Type: cross Abstract: Deploying Large Language Models (LLMs) directly on mobile platforms at the edge is gaining traction due to a myriad of benefits, such as increased pr...

By Weisi Yang, Stephen Xia

llms benchmarks safety

One email a morning, machine-written

Related stories

Resource-aware Computation-Communication Overlap for multi-GPU ML Workloads

A principled approach for energy-efficient training via phase-aware GPU frequency tuning

Local Inference of LLMs on Long Contexts: A Data Systems Perspective

CrossPool: Efficient Multi-LLM Serving for Cold MoE Models through KV-Cache and Weight Disaggregation

DualPace: Decoupled Scheduling and Cooperative GPU--CPU Decoding for LLM Serving

PELM: Power Efficient On-Device LLM Inference with Speculative Decoding and Dynamic Voltage Frequency Scaling