arXiv Machine Learning September 14, 2026 By Vaibhav Singh, Pierre-Andr\'e No\"el, Torsten Scholak, Eugene Belilovsky, Oleksiy Ostapenko

Fixed State, Long Reach: What a Constant-Size Cache Buys Block Diffusion at Scale

Read the original on arXiv Machine Learning →

The paper investigates how block‑diffusion language models can use a constant‑size cache to enable efficient parallel decoding. By employing sequence mixers that summarize completed blocks into a reusable state and a block‑causal training objective, the authors pretrain three 3B block‑diffusion denoisers (attention, Mamba, and hybrid) on 300 B tokens. The resulting state‑space cache remains O(1) in memory and latency regardless of context length, yielding significant speed‑up and memory savings compared to traditional attention‑based caches, especially at very long sequences.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 7

Training Hybrid Block Diffusion Language Models with Partial Bidirectionality

arXiv:2607. 02805v1 Announce Type: cross Abstract: High-throughput long-context generation is one of the central challenges for large language models.

By Pranshu Chaturvedi, Parth Shroff, Tarun Suresh, Hangoo Kang, Kaiyue Wen

llms diffusion efficiency

arXiv AI

Sep 1

DASC: Decay-Aware State Compression for Hybrid Linear-Attention Serving

arXiv:2608.30386v1 Announce Type: cross Abstract: Hybrid linear-attention architectures have recently scaled to large open-weight models, offering quality competitive with full attention while substa...

By Yanqi Yu, Pingwei Sun, Jianchao Tan, Tao Zhang, Yuchen Xie, Xunliang Cai, Yao Liu

efficiency benchmarks

arXiv Machine Learning

Aug 10

Retrofitting Linear Attention into Diffusion Language Models

arXiv:2608. 06628v1 Announce Type: new Abstract: Diffusion language models (dLLMs) offer a promising alternative to autoregressive models by accelerating inference through parallel decoding.

By Jinha Kim, Younghun Roh, Jaeyeon Kim

llms diffusion efficiency benchmarks

arXiv AI

Jul 21

FlowBlock: Wavefront-Parallel Decoding for Self-Correcting Diffusion Language Models

arXiv:2607. 17652v1 Announce Type: new Abstract: Block-wise diffusion large language models (dLLMs) decode sequentially at the block level, enabling effective KV-cache reuse across blocks but making inter-block decoding strictly serial.

By Bing Tian, Haikun Liu, Xiaocheng Zhong, Zhuohui Duan, Zhaokai Luo, Huayi Jin, Zhiyong Wang, Xiaofei Liao

llms diffusion efficiency benchmarks

arXiv AI

Jun 29

Bifocal Diffusion Language Models: Asymmetric Bidirectional Context for Parallel Generation

arXiv:2606. 27732v1 Announce Type: cross Abstract: Discrete diffusion language models (dLLMs) recover masked tokens in parallel, offering significant speedups over autoregressive (AR) generation.

By Yuhang Chen, Xianfeng Wu, Jinhao Duan, Mingfu Liang, Xiaohan Wei, Yunchen Pu, Fei Tian, Chonglin Sun, Parish Aggarwal, Frank Shyu, Luke Simon, Sandeep Pandey, Xi Liu, Tianlong Chen

llms diffusion efficiency benchmarks

Hugging Face Trending Papers

Jul 5

Sangam: Efficiently Serving Diffusion LLMs with the AR Stack

Diffusion language models (dLLMs) generate text by iteratively denoising a masked response and can commit multiple output positions per model invocation. Their bidirectional attention prevents exact autoregressive-style KV caching, since committing one position shifts the KV activations of all others.

llms diffusion

One email a morning, machine-written

Related stories

Training Hybrid Block Diffusion Language Models with Partial Bidirectionality

DASC: Decay-Aware State Compression for Hybrid Linear-Attention Serving

Retrofitting Linear Attention into Diffusion Language Models

FlowBlock: Wavefront-Parallel Decoding for Self-Correcting Diffusion Language Models

Bifocal Diffusion Language Models: Asymmetric Bidirectional Context for Parallel Generation

Sangam: Efficiently Serving Diffusion LLMs with the AR Stack