arXiv AI July 10, 2026 By Haozhan Tang, Zerui Wang, Yuxian Gu, Song Han, Han Cai

Jet-Long: Efficient Long-Context Extension with Dynamic Bifocal RoPE

arXiv:2607. 07740v1 Announce Type: cross Abstract: Modern LLMs are increasingly deployed in long-context applications such as retrieval-augmented generation, repository-level coding, and agentic workflows whose accumulated reasoning and tool traces routinely push the input an order of magnitude past the pretraining window, making zero-shot context extension the dominant deployment path for open-weight checkpoints.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI

Jul 23

LISA: Linear-Indexed Sparse Attention for Efficient Long-Context Reasoning

arXiv:2607. 19358v1 Announce Type: new Abstract: Recent advances in long chain-of-thought reasoning models such as DeepSeek-R1 have led to increasingly longer inference context lengths under the test-time scaling paradigm.

By Yu Zhao, Zekun Zhang, Fan Jiang, Bo Zeng, Linlong Xu, Shimin Shan, Yu Liu, Longyue Wang, Weihua Luo

efficiency benchmarks

arXiv Machine Learning

Aug 5

ATFlash: Per-RoPE-Wavelength Attention Windows for Compute/Memory-Efficient LLM Inference

arXiv:2608. 02947v1 Announce Type: new Abstract: The attention score with rotary position embeddings (RoPE) decomposes exactly into a sum over its 2D-rotation frequency pairs, and each pair's wavelength limits how far it can discriminate position.

By Shun-ichiro Hayashi, Daichi Mukunoki, Tetsuya Hoshino, Takahiro Katagiri

llms rag efficiency benchmarks

Hugging Face Trending Papers

Aug 3

ATFlash: Per-RoPE-Wavelength Attention Windows for Compute/Memory-Efficient LLM Inference

The attention score with rotary position embeddings (RoPE) decomposes exactly into a sum over its 2D-rotation frequency pairs, and each pair's wavelength limits how far it can discriminate position. Aligned with this structure, we propose the per-RoPE-wavelength distance window: it prunes the query--key inner-product terms beyond a wavelength-proportional distance.

llms rag efficiency benchmarks

arXiv AI

Jul 3

Multi-Head Recurrent Memory Agents

arXiv:2607. 01523v1 Announce Type: cross Abstract: Recurrent memory agents extend LLMs to arbitrarily long contexts by iteratively consolidating input into a fixed-size memory window.

By Jiatong Li, Samuel Yeh, Sharon Li

llms agents benchmarks

arXiv AI

Aug 5

$\pi$-Attention: Online Efficient Sparse Transformers for Long-Context Modeling

arXiv:2511. 10696v3 Announce Type: replace-cross Abstract: Sparse attention is crucial in long-context Transformers, which restricts each token to a limited neighborhood and thereby reduces the quadratic cost of full self-attention.

By Pike D. Liu, Chang Liu, Yanxuan Yu

llms

arXiv Machine Learning

Jun 9

Still: Amortized KV Cache Compaction in a Single Forward Pass

arXiv:2606. 07878v1 Announce Type: new Abstract: The KV cache is the memory bottleneck of long-horizon language model deployment.

By Charles O'Neill, Alex Sandomirsky, Harry Partridge, Mudith Jayasekara, Max Kirkby

llms nlp efficiency

Related stories

LISA: Linear-Indexed Sparse Attention for Efficient Long-Context Reasoning

ATFlash: Per-RoPE-Wavelength Attention Windows for Compute/Memory-Efficient LLM Inference

ATFlash: Per-RoPE-Wavelength Attention Windows for Compute/Memory-Efficient LLM Inference

Multi-Head Recurrent Memory Agents

$\pi$-Attention: Online Efficient Sparse Transformers for Long-Context Modeling

Still: Amortized KV Cache Compaction in a Single Forward Pass