arXiv AI
Aug 12
Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformers
arXiv:2604. 07822v2 Announce Type: replace-cross Abstract: We study implicit reasoning, i.
DiscoLoop: Looping Discrete Embeddings and Continuous Hidden States for Multi-hop Reasoning
arXiv:2607. 00341v1 Announce Type: cross Abstract: Large language models achieve strong performance on many reasoning tasks when allowed to externalize intermediate steps as Chain-of-Thought (CoT).
Related stories
arXiv Computation and Language
Aug 28
Why Knowing Both Hops Is Not Enough: Understanding Two-Hop Generalization in Language Models
Large language models can solve complex multi‑hop tasks but often fail on simple two‑hop queries, even when each hop is individually correct. In a controlled symbolic setting, the authors find that models generalize reliably when the second hop follows the training distribution, but always fail when it deviates. Mechanistic analysis shows that successful generalization relies on consistent intermediate representations across contexts, whereas failures arise from a mismatch between lower‑layer representation construction and upper‑layer mapping to outputs. The study proposes a recurrent‑style training strategy that improves out‑of‑distribution two‑hop generalization.
arXiv AI
Sep 16
Thinking Deeper, Not Longer: Memory-Efficient Test-Time Reasoning with Depth-Recurrent Transformers for Compositional Generalization
arXiv:2603.21676v2 Announce Type: replace-cross Abstract: Standard Transformers have a fixed computational depth, limiting their ability to generalize to tasks that require variable-depth reasoning....
arXiv AI
Jul 29
Penelope: Localized Latent Recurrence for Efficient Structured Reasoning
arXiv:2607. 25915v1 Announce Type: new Abstract: Complex structured reasoning tasks often require additional computation, yet current language models obtain it mainly by increasing parameter scale or by serializing intermediate steps as chain-of-thought (CoT) tokens.
Hugging Face Trending Papers
5d ago
Not All Thinking is Created Equal: Latent Reasoning Discovers a Recurrent Search Algorithm for Depth Generalization
The paper investigates whether different forms of intermediate computation in large language models—such as token-based traces, pause tokens, and latent reasoning—rely on the same underlying mechanism. By training five variants of GPTNeoX on an extended multi-hop reasoning task, the authors find that while vanilla, Chain-of-Thought, and Pause Token models perform well on in-distribution data, they fail to generalize to longer-hop out-of-distribution problems. In contrast, latent-reasoning models exhibit better depth generalization, with causal analysis revealing a sparse recurrent search circuit that implements forward reachability propagation across the graph.
arXiv AI
Jul 17
T^2MLR: Transformer with Temporal Middle-Layer Recurrence
arXiv:2607. 15178v1 Announce Type: cross Abstract: Transformer reasoning is limited by autoregressive decoding, which repeat edly compresses rich hidden computation through token space and makes it difficult for intermediate reasoning states to persist across time.
arXiv AI
Sep 1
Zipping the Thought: When and How Compressed Reasoning Data Works in LLM Post-Training
The paper investigates how different forms of compressed chain‑of‑thought (CoT) reasoning—Explicit, Composed, and Implicit—affect large language model (LLM) performance after supervised fine‑tuning (SFT). Using a synthetic compositional reasoning task, the authors show that coarser CoT requires more SFT data, that Composed and Implicit CoT benefit more from data scaling (with Composed also benefiting from repetition), and that reinforcement learning with verifiable rewards (RLVR) can decompose compressed steps learned during SFT. Additionally, unidirectional CoT ordering improves generalization on longer sequential tasks.
arXiv AI
Jun 9
Why Limit the Residual Stream to Layers and Not Tokens? Persistent Memory for Continuous Latent Reasoning
arXiv:2606. 07720v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated remarkable reasoning abilities on mathematical and multi-hop planning tasks.
arXiv AI
Sep 1
AgenticRag-R1: Agentic Reinforcement Learning with Stack Memory for Multi-Step Reasoning, Retrieval and Memorizing
AgenticRag‑R1 is a reinforcement‑learning framework that integrates reasoning, retrieval, and memory through a stack and fine‑grained action space. It uses hierarchical action‑aware rewards and an information‑aware trajectory rejection strategy to support long‑horizon learning. Experiments on multi‑hop, open‑domain, and agentic reasoning benchmarks show that AgenticRag‑R1 outperforms strong baselines and produces robust, interpretable, memory‑aware reasoning behaviors.
arXiv AI
Sep 10
A*-Thought-V2: Efficient Latent Reasoning via Geometric Dynamics of LLM
A*-Thought-V2 is a framework that models Chain-of-Thought reasoning as a geometric trajectory in a 3D PCA space, using explicit-implicit latent tokens to compress steps that deviate from the main question-to-solution direction. The method measures alignment angles to decide which steps remain text and which become latent, and introduces stepwise embedding forcing and label forcing to train the architecture. Experiments on Qwen models show up to 2.6% accuracy gains, halved response length, and significant reductions in computation and training time.
arXiv AI
Jun 15
Fractured Chain-of-Thought Reasoning
arXiv:2505. 12992v4 Announce Type: replace-cross Abstract: Inference-time scaling techniques have significantly bolstered the reasoning capabilities of large language models (LLMs) by harnessing additional computational effort at inference without retraining.
arXiv AI
Sep 3
CHASE: Cache-Hole-Adapted Skip Exit for Looped State-Space Language Models
The paper introduces CHASE, a cache‑hole‑adapted skip‑exit mechanism for looped state‑space language models, specifically Looped Mamba and Looped Hybrid Mamba‑Transformer. It shows that looping these architectures improves performance on controlled reasoning tasks and remains competitive in pre‑training benchmarks while using fewer distinct parameters. The cache‑hole adaptation allows selective skipping of recurrent steps during inference, maintaining perplexity close to full computation and achieving significant speedups.