arXiv AI
Jun 3
dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching
arXiv:2506. 06295v2 Announce Type: replace-cross Abstract: Autoregressive Models (ARMs) have long dominated the landscape of Large Language Models.
CoTA++: Understanding the Repeat Curse in dMLLMs from an Information Flow Perspective
Related stories
arXiv AI
Sep 1
Tail-Replay: Escaping the Curse of Linear Attention in Prefix Caching for Hybrid LLMs
arXiv:2608.30310v1 Announce Type: cross Abstract: Hybrid large language models interleave full-attention layers with linear-attention layers to reduce the cost of long-context inference. This structu...
arXiv Machine Learning
Aug 7
QEvict: Recoverable Quantized KV Eviction for Attention-Drift-Robust Long-Context Decoding
arXiv:2608. 05326v1 Announce Type: new Abstract: Autoregressive large language model inference is increasingly constrained by the memory footprint of the Key-Value (KV) cache.
arXiv Machine Learning
Sep 16
Window-Diffusion: Accelerating Diffusion Language Model Inference with Windowed Token Pruning and Caching
The paper introduces Window-Diffusion, a method that accelerates diffusion language model inference by pruning and caching tokens within a sliding window. It categorizes undecoded tokens into active, buffer, and far-field groups, computing only the first two while discarding the rest. Experiments on LLaDA and Dream demonstrate up to 99× speedup with minimal loss in generation quality.
arXiv Computation and Language
Aug 21
Remask, Don't Replace: Token-to-Mask Refinement in Diffusion Language Models
arXiv:2604. 18738v3 Announce Type: replace Abstract: Diffusion language models (dLLMs) generate text through iterative denoising, filling multiple masked positions at each step.
arXiv Machine Learning
Aug 28
Affix Cache for Diffusion Large Language Models
The paper introduces ACache, an affix-oriented cache reuse mechanism for Diffusion Large Language Models (DLLMs). ACache identifies a small set of critical affix tokens, called Anchor Tokens, and selectively recomputes their key-value states while reusing the rest of the affix cache. Experiments on Fast-dLLM and Nano-vLLM show that recomputing about 20% of affix tokens restores accuracy and can reduce recompute latency by up to 55.7% while improving throughput by up to 1.68×.
arXiv Machine Learning
Sep 14
Fixed State, Long Reach: What a Constant-Size Cache Buys Block Diffusion at Scale
The paper investigates how block‑diffusion language models can use a constant‑size cache to enable efficient parallel decoding. By employing sequence mixers that summarize completed blocks into a reusable state and a block‑causal training objective, the authors pretrain three 3B block‑diffusion denoisers (attention, Mamba, and hybrid) on 300 B tokens. The resulting state‑space cache remains O(1) in memory and latency regardless of context length, yielding significant speed‑up and memory savings compared to traditional attention‑based caches, especially at very long sequences.
arXiv Machine Learning
Jun 26
Dynamic-dLLM: Dynamic Cache-Budget and Adaptive Parallel Decoding for Training-Free Acceleration of Diffusion LLM
arXiv:2606. 26120v1 Announce Type: cross Abstract: Diffusion Large Language Models (dLLMs) offer a promising alternative to autoregressive models, excelling in text generation tasks due to their bidirectional attention mechanisms.
arXiv Machine Learning
Jun 2
MomentKV: Closing the Directional Gap in KV Cache Eviction for Long-Context Inference
arXiv:2606. 01563v1 Announce Type: new Abstract: Autoregressive decoding in Transformer-based language models relies on the KV cache, whose memory footprint grows linearly with sequence length and becomes the primary bottleneck for long-context inference.
arXiv AI
Jul 21
LaCache: Exact Caching and Precision-Adaptive Inference for Diffusion Large Language Models
arXiv:2607. 16339v1 Announce Type: new Abstract: Diffusion-based Large Language Models(DLLMs) enable parallel generation via Semi-Autoregressive (SAR) decoding in text generation.
arXiv AI
Jun 15
Residual Context Diffusion Language Models
arXiv:2601. 22954v2 Announce Type: replace-cross Abstract: Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to purely autoregressive language models because they can decode multiple tokens in parallel.
arXiv Machine Learning
Jul 31
Back from the Future: Key-Value Cache Management by Counter-Causal Surprise
arXiv:2607. 27600v1 Announce Type: new Abstract: Key-value (KV) cache management through compression and eviction strategies has emerged as an important research direction in recent years.