arXiv Machine Learning June 10, 2026 By Muhammad Ahmed

Parallel Causal Associative Fields: Gated Sparse Memory for Long-Context Language Modeling

Read the original on arXiv Machine Learning →

arXiv:2606. 10435v1 Announce Type: new Abstract: Transformers achieve strong language modeling performance by providing direct token-to-token communication paths, but causal self-attention scales quadratically with context length.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

llms

arXiv Machine Learning

Jun 30

Memory-Managed Long-Context Attention: A Preliminary Study of Editable Request-Local Memory

arXiv:2606. 28876v1 Announce Type: cross Abstract: Long-context language models often conflate two different goals: compressing history into an efficient state, and maintaining reliable long-term memory.

By Junyi Zou, Avrova Donz

llms benchmarks

arXiv AI

2d ago

BCMT: Blockwise Causal Memory Transformer

arXiv:2608. 13578v1 Announce Type: cross Abstract: Transformer architectures rely on dense self-attention to model long-range dependencies, but this mechanism exhibits quadratic complexity with respect to sequence length.

By Rachid Arezki

llms efficiency benchmarks

arXiv AI

Jun 16

Long-Context Modeling via GSS-Transformer Hybrid Architecture with Learnable Mixing

arXiv:2606. 16093v1 Announce Type: cross Abstract: Modeling long-range dependencies remains a central challenge in natural language processing.

By Kuzey Torlak, H\"useyin Arda Arslan, An{\i}l Dervi\c{s}o\u{g}lu, Beyza Nur Deniz, Onur Boyar

llms nlp efficiency

arXiv Machine Learning

Jul 13

Memory-Managed Long-Context Attention: Bounded Editable Memory with a Hard Lifecycle and Calibrated Sparse Fallback

arXiv:2606. 28876v2 Announce Type: replace-cross Abstract: We study memory-managed long-context attention: explicit bounded memory with a learned query-independent writer, lifecycle control, query-aware reading, calibrated sparse fallback, and frozen-LLM generation from raw evidence.

By Junyi Zou, Avrova Donz

llms

arXiv AI

Jun 29

Bifocal Diffusion Language Models: Asymmetric Bidirectional Context for Parallel Generation

arXiv:2606. 27732v1 Announce Type: cross Abstract: Discrete diffusion language models (dLLMs) recover masked tokens in parallel, offering significant speedups over autoregressive (AR) generation.

By Yuhang Chen, Xianfeng Wu, Jinhao Duan, Mingfu Liang, Xiaohan Wei, Yunchen Pu, Fei Tian, Chonglin Sun, Parish Aggarwal, Frank Shyu, Luke Simon, Sandeep Pandey, Xi Liu, Tianlong Chen

llms diffusion efficiency benchmarks

arXiv Machine Learning

Jul 7

Learning When to Attend: Conditional Memory Access for Long-Context LLMs

arXiv:2603. 17484v2 Announce Type: replace-cross Abstract: Language models struggle to generalize beyond pretraining context lengths, limiting long-horizon reasoning and retrieval.

By Sakshi Choudhary, Aditya Chattopadhyay, Luca Zancato, Elvis Nunez, Matthew Trager, Wei Xia, Stefano Soatto

llms efficiency

Related stories

Memory-Managed Long-Context Attention: A Preliminary Study of Editable Request-Local Memory

BCMT: Blockwise Causal Memory Transformer

Long-Context Modeling via GSS-Transformer Hybrid Architecture with Learnable Mixing

Memory-Managed Long-Context Attention: Bounded Editable Memory with a Hard Lifecycle and Calibrated Sparse Fallback

Bifocal Diffusion Language Models: Asymmetric Bidirectional Context for Parallel Generation

Learning When to Attend: Conditional Memory Access for Long-Context LLMs