arXiv AI
Jul 7
Training Hybrid Block Diffusion Language Models with Partial Bidirectionality
arXiv:2607. 02805v1 Announce Type: cross Abstract: High-throughput long-context generation is one of the central challenges for large language models.
MAGE: All-[MASK] Block Already Knows Where to Look in Block Diffusion LLM
arXiv:2602. 14209v2 Announce Type: replace Abstract: Block diffusion LLMs are an emerging paradigm for parallel language generation, but their KV caching makes memory access the dominant bottleneck in long-context inference.
Related stories
arXiv Machine Learning
Aug 10
Retrofitting Linear Attention into Diffusion Language Models
arXiv:2608. 06628v1 Announce Type: new Abstract: Diffusion language models (dLLMs) offer a promising alternative to autoregressive models by accelerating inference through parallel decoding.
arXiv AI
Jul 21
Lil: Less is Less When Applying Post-Training Sparse-Attention Algorithms in Long-Decode Stage
arXiv:2601. 03043v4 Announce Type: replace-cross Abstract: Large language models (LLMs) demonstrate strong capabilities across a wide range of complex tasks and are increasingly deployed at scale, placing significant demands on inference efficiency.
arXiv AI
Jun 9
Full Attention Strikes Back: Transferring Full Attention into Sparse within Hundred Training Steps
arXiv:2605. 16928v2 Announce Type: replace-cross Abstract: Long-context inference in large language models is bottlenecked by the quadratic cost of full attention.
arXiv AI
Jul 7
IndexMem: Learned KV-Cache Eviction with Latent Memory for Long-Context LLM Inference
arXiv:2605. 25475v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are increasingly expected to operate over long contexts, yet standard softmax attention incurs a KV cache that grows linearly with sequence length, quickly becoming the bottleneck for long context inference.
arXiv AI
Jul 28
Beyond Block Boundaries: Multi-Block Editing for Diffusion Large Language Models
arXiv:2607. 22663v1 Announce Type: new Abstract: Block diffusion has emerged as the dominant paradigm for scaling discrete diffusion language models (dLLMs), because decoding text in fixed-size blocks preserves parallel generation within each block while keeping the quadratic attention cost tractable.
arXiv AI
Aug 5
PI-Mem: Pushing Long-Context Reasoning to 3.6M Tokens with Parallel-Iterative Memory
arXiv:2608. 03048v1 Announce Type: cross Abstract: Long-context reasoning remains a critical bottleneck for large language models, as recent recurrent-memory approaches face two inherent challenges: sequential chunk-wise updates can overwrite early critical evidence with later irrelevant content, and serial inter-chunk dependencies limit parallelism and cause latency to increase with context length.
arXiv Machine Learning
Jul 2
FSA: An Alternative Efficient Implementation of Native Sparse Attention Kernel
arXiv:2508. 18224v3 Announce Type: replace-cross Abstract: Recent advances in sparse attention mechanisms have demonstrated strong potential for reducing the computational cost of long-context training and inference in large language models (LLMs).
arXiv AI
Jul 24
Understanding and Accelerating the Training of Masked Diffusion Language Models
arXiv:2605. 13026v2 Announce Type: replace-cross Abstract: Masked diffusion models (MDMs) have emerged as a promising alternative to autoregressive models (ARMs) for language modeling.
arXiv AI
Jun 30
Coverage-Driven KV Cache Eviction for Efficient and Improved Inference of LLM
arXiv:2606. 29563v1 Announce Type: cross Abstract: Large language models (LLMs) excel at complex tasks like question answering and summarization, thanks to their ability to handle long-context inputs.
arXiv Machine Learning
Jul 7
Learning When to Attend: Conditional Memory Access for Long-Context LLMs
arXiv:2603. 17484v2 Announce Type: replace-cross Abstract: Language models struggle to generalize beyond pretraining context lengths, limiting long-horizon reasoning and retrieval.
arXiv AI
6d ago
LoSA: Near-Lossless Sparse Attention for Training-Free Video Diffusion Acceleration
arXiv:2608. 12032v1 Announce Type: cross Abstract: Video diffusion transformers are costly to sample: every denoising step applies self-attention over a long 3D token sequence, a quadratic cost that dominates as resolution and duration grow.