arXiv Machine Learning July 13, 2026 By Yuanjie Zhu, Liangwei Yang, Ke Xu, Weizhi Zhang, Shanghao Li, Zihe Song, Philip S. Yu

BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving

Read the original on arXiv Machine Learning →

arXiv:2607. 08930v1 Announce Type: new Abstract: Efficient serving of diffusion large language models (dLLMs) is hindered by convergence heterogeneity: when batching multiple requests, different sequences converge at different rates, causing faster requests to stall behind slower stragglers and introducing compute bubbles and tail latency.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

Hugging Face Trending Papers

Jul 9

BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving

Efficient serving of diffusion large language models (dLLMs) is hindered by convergence heterogeneity: when batching multiple requests, different sequences converge at different rates, causing faster requests to stall behind slower stragglers and introducing compute bubbles and tail latency. We present BlockServe, a continuous batching framework that integrates block-grained scheduling -- immediately evicting completed requests at block boundaries -- with mixed-state execution that extends dual cache and parallel decoding to heterogeneous batches via gather-scatter indexing.

llms diffusion efficiency benchmarks

arXiv Machine Learning

Jul 7

Sangam: Efficiently Serving Diffusion LLMs with the AR Stack

arXiv:2607. 04206v1 Announce Type: cross Abstract: Diffusion language models (dLLMs) generate text by iteratively denoising a masked response and can commit multiple output positions per model invocation.

By Nitin Kedia, Saurabh Agarwal, Myungjin Lee, Aditya Akella

llms diffusion

Hugging Face Trending Papers

Jul 5

Sangam: Efficiently Serving Diffusion LLMs with the AR Stack

Diffusion language models (dLLMs) generate text by iteratively denoising a masked response and can commit multiple output positions per model invocation. Their bidirectional attention prevents exact autoregressive-style KV caching, since committing one position shifts the KV activations of all others.

llms diffusion

arXiv Machine Learning

Jun 30

DiLaServe: High SLO Attainment Serving for Diffusion Language Models

arXiv:2606. 29094v1 Announce Type: new Abstract: Diffusion language models (DLMs) have recently emerged as a promising alternative to conventional autoregressive language models.

By Tzu-Tao Chang, Benjamin Yuanyang Hong, Kiet Pham, Shivaram Venkataraman

llms diffusion efficiency benchmarks

arXiv AI

Jun 2

BlockBatch: Multi-Scale Consensus Decoding for Efficient Diffusion Language Model Inference

arXiv:2605. 29233v2 Announce Type: replace-cross Abstract: Diffusion language models (dLLMs) generate text by iteratively denoising multiple token positions in parallel, offering an attractive alternative to strictly autoregressive decoding.

By Xiaoyou Wu, Cheng-Jhih Shih, Binfei Ji, Yong Liu, Yingyan Celine Lin

llms diffusion

arXiv AI

Jun 2

Threshold-Based Exclusive Batching for LLM Inference

arXiv:2606. 00516v1 Announce Type: new Abstract: Mixed batching (MB)--interleaving prefill and decode in a single batch--has become the standard scheduling strategy for large language model (LLM) inference due to its efficiency in maximizing compute and memory utilization.

By Weifang Zhang, Yuzhou Nie, Bowen Pang, Guangrui Ma, Shining Wu

llms efficiency

Related stories

BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving

Sangam: Efficiently Serving Diffusion LLMs with the AR Stack

Sangam: Efficiently Serving Diffusion LLMs with the AR Stack

DiLaServe: High SLO Attainment Serving for Diffusion Language Models

BlockBatch: Multi-Scale Consensus Decoding for Efficient Diffusion Language Model Inference

Threshold-Based Exclusive Batching for LLM Inference