arXiv Machine Learning October 8, 2026 By Huishan Ji, Hua Xu, Weiming Zhang, Qirui Ye

YANchor-4B: Effective Long-Horizon Reasoning in O(N) Time with O(1) Memory

Read the original on arXiv Machine Learning →

YANchor-4B is a recurrent model that achieves long-horizon reasoning in linear time while using constant memory by preserving key information as anchors for later retrieval. It outperforms linear-time, constant-state baselines on challenging math benchmarks, achieving 82.93% mean pass@1 on AIME 2024–2026 and 63.64% on HMMT. The model also provides several-fold higher batched long-generation throughput than Transformer and hybrid baselines on H100, and demonstrates superior general-purpose capabilities across dozens of benchmarks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Aug 5

PI-Mem: Pushing Long-Context Reasoning to 3.6M Tokens with Parallel-Iterative Memory

arXiv:2608. 03048v1 Announce Type: cross Abstract: Long-context reasoning remains a critical bottleneck for large language models, as recent recurrent-memory approaches face two inherent challenges: sequential chunk-wise updates can overwrite early critical evidence with later irrelevant content, and serial inter-chunk dependencies limit parallelism and cause latency to increase with context length.

By Dawei Liu, Haixu Song, Shuang Cheng, Shijie Wang, Haozheng Hou, Kaifeng Liu, Ermo Hua, Zhonghang Yuan, Zhijie Zhong, Yuchen Fan, Biqing Qi, Bowen Zhou

llms nlp reinforcement-learning benchmarks

arXiv AI

Oct 15

History Replays Itself: Unleashing Long-Context Capability in Linear Attention

arXiv:2610.15715v1 Announce Type: new Abstract: Linear attention promises the efficiency needed for long-context language modelling, yet its fixed-size recurrent state must simultaneously support ong...

By Li Nanbo, Firas Laakom, Yilan Zhang, Piotr Pi\k{e}kos, Yimeng Chen, Liangyu Wang, Yifan Xu, Mateusz Ostaszewski, Dandan Guo, Wenyi Wang, Yanning Dai, Yuhui Wang, J\"{u}rgen Schmidhuber

llms rag efficiency

arXiv Computation and Language

Aug 24

Memory Augmentation Unlocks Efficient Chain-of-Thought Reasoning

arXiv:2608.21265v1 Announce Type: new Abstract: Large language models often rely on Chain-of-Thought (CoT) reasoning to solve complex tasks, but verbose reasoning traces introduce substantial inferen...

By Simeng Zhang, Yilong Chen, Wenyuan Zhang, Zhenyu Zhang, Yao Chen, Junyuan Shang, Tingwen Liu

llms nlp benchmarks

arXiv AI

Jul 29

Penelope: Localized Latent Recurrence for Efficient Structured Reasoning

arXiv:2607. 25915v1 Announce Type: new Abstract: Complex structured reasoning tasks often require additional computation, yet current language models obtain it mainly by increasing parameter scale or by serializing intermediate steps as chain-of-thought (CoT) tokens.

By Yutong Chen, Shouqian Shi, Xinran Liu, Haochen Wang, Jiaying Wang, Tianxing Xu, Yuanxi Wang, Zirui Ding

llms diffusion efficiency benchmarks

arXiv Machine Learning

Aug 14

MARCH: Scaling Recurrent Memory with Content-Routed State Anchors

arXiv:2608. 12435v1 Announce Type: new Abstract: Transformers owe much of their strong long-context retrieval capability to a token-level memory that grows with context length.

By Ming Zhang, Kaisen Yang, Shu Yu, Ermo Hua, Ning Ding, Xia Hu, Bowen Zhou, Chaochao Lu, Youbang Sun

llms

arXiv Machine Learning

Sep 7

BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference

BeaconKV is a training‑free key‑value cache compression technique for Large Reasoning Models that uses beacon queries—compact representatives of query clusters—to predict which KV pairs will be revisited during long‑horizon reasoning. By focusing on Thought Revisiting Tokens that re‑attend distant context, BeaconKV reduces memory usage up to 5.8× and improves throughput by over 4.3× while largely preserving cache accuracy across multiple open‑source LRMs and reasoning benchmarks.

By Janghyeon Kim, Minsoo Kim, Kyuhong Shim, Jungwook Choi

rag efficiency benchmarks

One email a morning, machine-written

Related stories

PI-Mem: Pushing Long-Context Reasoning to 3.6M Tokens with Parallel-Iterative Memory

History Replays Itself: Unleashing Long-Context Capability in Linear Attention

Memory Augmentation Unlocks Efficient Chain-of-Thought Reasoning

Penelope: Localized Latent Recurrence for Efficient Structured Reasoning

MARCH: Scaling Recurrent Memory with Content-Routed State Anchors

BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference