The paper proposes a simple technique of chunking workloads into smaller parts that alternate between compute-intensive and memory-bound operations to smooth power and temperature spikes in GPU systems. By doing so, it prevents throttling, leading to faster wall-clock times and lower total energy consumption. Experiments on a DGX Spark show up to 2% performance and energy gains, while similar benefits, though smaller, are observed on multi‑GPU servers.
arXiv:2606. 09200v1 Announce Type: cross Abstract: The rapid growth of large-scale machine learning (ML) has made distributed training across multiple GPUs a fundamental component of modern ML systems.
arXiv:2609.24205v1 Announce Type: cross Abstract: Modern AI model training imposes unprecedented computational demands, making it a key contributor to datacenter energy consumption. Yet a significant...
arXiv:2610.28223v1 Announce Type: cross Abstract: Large language models can solve complex problems across a variety of subjects, such as mathematics and coding. For such complex analytical tasks, mod...
arXiv:2606. 24506v1 Announce Type: cross Abstract: Emerging LLM services increasingly host many sparse MoE models, yet most models receive sparse requests and remain cold.
DualPace is an LLM serving runtime that decouples scheduling between a GPU engine and an AMX‑enabled CPU engine, allowing the CPU to independently batch and advance decodes while still cooperating with the GPU for dense stages. This design expands key‑value cache capacity into host memory and reduces bursty CPU progress, leading to significant throughput gains—up to 43.2 % over NEO and 90.6 % over vLLM—without increasing latency. The system also maintains low time‑to‑first‑token and supports higher request rates at comparable tail latency.
arXiv:2609.09662v1 Announce Type: cross Abstract: Deploying Large Language Models (LLMs) directly on mobile platforms at the edge is gaining traction due to a myriad of benefits, such as increased pr...
arXiv:2602. 24044v2 Announce Type: replace-cross Abstract: Large Language Model (LLM) adapters enable low-cost model specialization, but introduce complex caching and scheduling challenges in distributed serving systems where hundreds of adapters must be hosted concurrently.
Large Language Model (LLM) inference workloads are a rapidly growing contributor to data center energy consumption. Optimizing these deployments requires matching specific LLMs to the most efficient GPUs, but operators currently lack the tools to do so without exhaustively profiling each combination.
arXiv:2607. 22614v1 Announce Type: new Abstract: RL-based LLM post-training increasingly disaggregates Rollout and Training across separate GPU resources, but static GPU partitioning suffers from severe pipeline bubbles under long-tail rollout latency.
The paper investigates how well common large‑language‑model (LLM) inference optimizations perform on Apple Silicon compared to server‑class NVIDIA GPUs. Four optimization families—KV‑cache pruning, speculative decoding, ported GPU kernels, and post‑training quantization—are evaluated on four Apple machines under a single‑request decode workload, with matched RTX 4090 comparisons where possible. Results show that KV‑cache pruning can boost Apple throughput up to 1.75×, while speculative decoding actually reduces throughput on Apple; CUDA kernel designs transfer only partially, and quantization gains vary by machine, indicating that speedups depend on the specific technique, runtime, hardware, and workload regime.
arXiv:2603. 16428v2 Announce Type: replace-cross Abstract: Fine-tuning Large Language Models (LLMs) has become essential for domain adaptation, but its memory-intensive property exceeds the capabilities of most GPUs.
arXiv:2607. 02391v1 Announce Type: cross Abstract: Large Language Model (LLM) inference workloads are a rapidly growing contributor to data center energy consumption.