Reasoning over infinitely long documents is becoming a reality with recurrent memory agents. However, increasing context length exacerbates the trade-off between reasoning performance and inference ef...
arXiv:2608. 03048v1 Announce Type: cross Abstract: Long-context reasoning remains a critical bottleneck for large language models, as recent recurrent-memory approaches face two inherent challenges: sequential chunk-wise updates can overwrite early critical evidence with later irrelevant content, and serial inter-chunk dependencies limit parallelism and cause latency to increase with context length.
arXiv:2607. 02509v1 Announce Type: new Abstract: Understanding and reasoning over long contexts has become a key requirement for deploying large language models (LLMs) in realistic applications.
Understanding and reasoning over long contexts has become a key requirement for deploying large language models (LLMs) in realistic applications. Although recent LLMs support increasingly long context windows, they often fail to use relevant evidence that is already present in the input, revealing a gap between context access and effective context utilization.
MemReread is a new agentic long‑context reasoning method that uses memory‑guided rereading instead of intermediate retrieval. By decomposing questions and selectively rereading when the final memory is insufficient, it recovers indirect evidence that might otherwise be lost during memory overwriting. A reinforcement‑learning framework further improves length extrapolation and adaptively controls the number of rereading passes, allowing MemReread‑4B to outperform strong baselines on long‑context tasks while keeping linear time complexity.
arXiv:2505. 17315v2 Announce Type: replace Abstract: Recent language models exhibit strong reasoning capabilities, yet the influence of long-context capacity on reasoning remains underexplored.
arXiv:2610.30132v1 Announce Type: cross Abstract: Reinforcement learning (RL) is an effective way to strengthen the long-context reasoning of large language models (LLMs), but it needs training data...
CoEM introduces a Commit-on-Evidence Memory system that learns when to compress source evidence into compact memory facts while preserving potentially useful excerpts verbatim in a pending set. The system uses a learned policy to decide whether to promote, retain, or discard each pending excerpt as new context arrives, and a frozen verifier ensures only supported facts are committed. Reinforcement learning trains this policy with step-level evidence rewards and final answer rewards, leading to consistent improvements in long-context reasoning, achieving 10.4–11.4 F1 points over the strongest baseline on 6,400-document inputs.
LatentHarness unifies memory access and latent reasoning by treating them as sequential latent actions—THINK, RECALL, and EXIT—within a language model. It is trained via counterfactual policy distillation, which evaluates the impact of each action on the emitted token and learns when to recall evidence versus continue reasoning. On six long‑context reasoning benchmarks, a 1.4B‑parameter LatentHarness model outperforms the strongest baselines by 2.8% and 10.0% relative, while running 5.9× faster than the leading long‑context baseline.
arXiv:2609.10441v1 Announce Type: cross Abstract: While Large Language Models (LLMs) have demonstrated impressive capabilities, they often struggle with extremely long contexts due to fixed context l...
arXiv:2507.06415v3 Announce Type: replace-cross Abstract: Long-context reasoning requires accurately identifying relevant information in extensive, noisy input contexts. In this work, we propose PERK...
While Large Language Models (LLMs) have demonstrated impressive capabilities, they often struggle with extremely long contexts due to fixed context limits. To address this, sequential approaches like...