arXiv AI August 26, 2026 By Farhana Amin, Sabiha Afroz, Mona Moghadampanah, Dimitrios S. Nikolopoulos

Serving Masked Diffusion LLMs: Characterization and Design Principles from Real Hardware

Masked diffusion language models (dLLMs) promise faster text generation by denoising multiple tokens simultaneously, yet their real‑world serving behavior has been largely unexamined. Using LLaDA‑8B‑Instruct on a single NVIDIA H200 GPU, the study finds that request difficulty is discretized into 11 step‑count levels, short‑budget benchmarks underestimate serving variance, and only 24% of single‑request time is GPU computation, with batching mainly reducing CPU dispatch overhead. The authors also demonstrate that output quality remains stable across batch sizes and propose a batch‑timeout rule for synchronized batching under Poisson arrivals.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning

Jul 7

Sangam: Efficiently Serving Diffusion LLMs with the AR Stack

arXiv:2607. 04206v1 Announce Type: cross Abstract: Diffusion language models (dLLMs) generate text by iteratively denoising a masked response and can commit multiple output positions per model invocation.

By Nitin Kedia, Saurabh Agarwal, Myungjin Lee, Aditya Akella

llms diffusion

arXiv Machine Learning

Aug 4

HERALD: High-Throughput Block Diffusion LLM Serving via CPU-GPU Cooperative KV Cache Retrieval

arXiv:2606. 21633v2 Announce Type: replace Abstract: The KV cache dominates GPU memory in long-context LLM serving, crowding out batch capacity and leaving GPU compute idle.

By Omin Kwon, Doyeon Kim, Jongseok Park, Seung Yul Lee, Ion Stoica, Jae W. Lee

llms diffusion efficiency benchmarks

arXiv Machine Learning

Aug 31

Characterization of Request and Token Energy Costs for LLM Inference Workloads on GPU Platforms

arXiv:2608.28044v1 Announce Type: cross Abstract: Large language model (LLM) inference serving is priced by tokens, but GPU energy is consumed over inference windows. This accounting mismatch makes t...

By Prabhu Vellaisamy, Vanessa Lam, Shawn Blanton, John Paul Shen

llms

Hugging Face Trending Papers

Jul 5

Sangam: Efficiently Serving Diffusion LLMs with the AR Stack

Diffusion language models (dLLMs) generate text by iteratively denoising a masked response and can commit multiple output positions per model invocation. Their bidirectional attention prevents exact autoregressive-style KV caching, since committing one position shifts the KV activations of all others.

llms diffusion

arXiv Machine Learning

Jul 13

BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving

arXiv:2607. 08930v1 Announce Type: new Abstract: Efficient serving of diffusion large language models (dLLMs) is hindered by convergence heterogeneity: when batching multiple requests, different sequences converge at different rates, causing faster requests to stall behind slower stragglers and introducing compute bubbles and tail latency.

By Yuanjie Zhu, Liangwei Yang, Ke Xu, Weizhi Zhang, Shanghao Li, Zihe Song, Philip S. Yu

llms diffusion efficiency benchmarks

arXiv Machine Learning

Aug 21

RequestRouter: Request-Boundary Routing for Efficient Single-GPU LLM Inference

arXiv:2605. 23057v2 Announce Type: replace Abstract: RequestRouter is a lightweight request-boundary controller for reducing the latency and energy cost of single-GPU large language model inference.

By Aman Sunesh, Ali Alshehhi, Hivansh Dhakne

llms efficiency benchmarks

One email a morning, machine-written

Related stories

Sangam: Efficiently Serving Diffusion LLMs with the AR Stack

HERALD: High-Throughput Block Diffusion LLM Serving via CPU-GPU Cooperative KV Cache Retrieval

Characterization of Request and Token Energy Costs for LLM Inference Workloads on GPU Platforms

Sangam: Efficiently Serving Diffusion LLMs with the AR Stack

BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving

RequestRouter: Request-Boundary Routing for Efficient Single-GPU LLM Inference