arXiv AI
Jul 7
Training Hybrid Block Diffusion Language Models with Partial Bidirectionality
arXiv:2607. 02805v1 Announce Type: cross Abstract: High-throughput long-context generation is one of the central challenges for large language models.
Bifocal Diffusion Language Models: Asymmetric Bidirectional Context for Parallel Generation
arXiv:2606. 27732v1 Announce Type: cross Abstract: Discrete diffusion language models (dLLMs) recover masked tokens in parallel, offering significant speedups over autoregressive (AR) generation.
Related stories
arXiv Machine Learning
Sep 14
Fixed State, Long Reach: What a Constant-Size Cache Buys Block Diffusion at Scale
The paper investigates how block‑diffusion language models can use a constant‑size cache to enable efficient parallel decoding. By employing sequence mixers that summarize completed blocks into a reusable state and a block‑causal training objective, the authors pretrain three 3B block‑diffusion denoisers (attention, Mamba, and hybrid) on 300 B tokens. The resulting state‑space cache remains O(1) in memory and latency regardless of context length, yielding significant speed‑up and memory savings compared to traditional attention‑based caches, especially at very long sequences.
arXiv AI
Jul 21
FlowBlock: Wavefront-Parallel Decoding for Self-Correcting Diffusion Language Models
arXiv:2607. 17652v1 Announce Type: new Abstract: Block-wise diffusion large language models (dLLMs) decode sequentially at the block level, enabling effective KV-cache reuse across blocks but making inter-block decoding strictly serial.
Hugging Face Trending Papers
Jul 5
Sangam: Efficiently Serving Diffusion LLMs with the AR Stack
Diffusion language models (dLLMs) generate text by iteratively denoising a masked response and can commit multiple output positions per model invocation. Their bidirectional attention prevents exact autoregressive-style KV caching, since committing one position shifts the KV activations of all others.
arXiv Machine Learning
Jul 7
Sangam: Efficiently Serving Diffusion LLMs with the AR Stack
arXiv:2607. 04206v1 Announce Type: cross Abstract: Diffusion language models (dLLMs) generate text by iteratively denoising a masked response and can commit multiple output positions per model invocation.
Hugging Face Trending Papers
Jul 20
FlowBlock: Wavefront-Parallel Decoding for Self-Correcting Diffusion Language Models
Block-wise diffusion large language models (dLLMs) decode sequentially at the block level, enabling effective KV-cache reuse across blocks but making inter-block decoding strictly serial. Prior work has attempted to unlock inter-block parallelism through post-training methods, but achieves only modest speedups and often degrades accuracy.
arXiv Machine Learning
Aug 10
Retrofitting Linear Attention into Diffusion Language Models
arXiv:2608. 06628v1 Announce Type: new Abstract: Diffusion language models (dLLMs) offer a promising alternative to autoregressive models by accelerating inference through parallel decoding.
arXiv Machine Learning
Jun 10
Parallel Causal Associative Fields: Gated Sparse Memory for Long-Context Language Modeling
arXiv:2606. 10435v1 Announce Type: new Abstract: Transformers achieve strong language modeling performance by providing direct token-to-token communication paths, but causal self-attention scales quadratically with context length.
arXiv AI
Sep 15
Self-Orchestrating Language Models: Leveraging Semantic Dependence for Efficient Inference
The paper introduces self‑orchestrating language models that annotate semantic dependence—identifying which tokens rely on others—to guide efficient inference. By leveraging these annotations, the authors design runtimes that parallelize autoregressive decoding, evict intermediate context, or determine denoising orders, achieving Pareto‑optimal quality‑efficiency trade‑offs. Three systems—PASTA, TIP, and Planned Diffusion—demonstrate these techniques for parallel decoding, memory‑efficient reasoning, and efficient discrete diffusion, respectively.
Hugging Face Trending Papers
Jul 23
Windowed-MTP: Removing the Full-Context Draft-KV Tax at Million-Token Context
Speculative decoding accelerates autoregressive generation by having a cheap draft propose tokens that a target verifies in parallel. Frontier models increasingly ship a built-in Multi-Token-Prediction (MTP/NEXTN) draft head under the assumption that the draft is negligibly cheap.
arXiv AI
Sep 3
Language Models Can Control Their Own Attention
The paper introduces Declarative Attention (DA), a protocol that lets language models explicitly declare which parts of their context to focus on during generation. By partitioning decoding into full-context, region-specific, and recent-output-only modes, the inference engine can skip large portions of the KV cache, dramatically reducing attended tokens. Experiments on 15 long-context tasks with off-the-shelf models show significant savings (52.0% and 31.1% reductions) with only modest accuracy drops that diminish as model size increases.
arXiv Machine Learning
Jul 24
Windowed-MTP: Removing the Full-Context Draft-KV Tax at Million-Token Context
arXiv:2607. 21535v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive generation by having a cheap draft propose tokens that a target verifies in parallel.