arXiv Machine Learning

RecurTrace: Adaptive Latent Reasoning with Loop-Time Memory

RecurTrace introduces adaptive latent reasoning for language models by allowing each looped layer to attend to its own past states and by using a halting head to decide when to stop iterating. This approach overcomes two limitations of prior latent recurrence methods: limited access to earlier computations and a fixed loop count that mismatches input difficulty. In experiments on MathQA, RecurTrace achieves 56.9% accuracy with an average of 2.0 loops, outperforming fixed‑depth baselines and other adaptive methods, and it also improves generation accuracy across a range of model sizes.

arXiv Computation and Language
Oct 12

One Model to Loop Them All: Elastic Latent Recurrence for Language Models

The paper introduces Elastic Latent Recurrence (ELR), a training recipe that enables looped language models to match the accuracy of non-recurrent models at a single loop while allowing additional loops to improve accuracy with more compute. ELR achieves this through gated state updates with learned step sizes, mixed loop counts during training, and conditioning on loop index and budget. The method also reduces training and inference overhead in Mixture-of-Experts models and produces a single checkpoint, Koi‑4B, that performs competitively with state‑of‑the‑art models on multiple benchmarks.

By Faisal Ladhak, Shantanu Acharya, Santiago Akle Serano, Simeng Sun, Boris Ginsburg, Krishna C Puvvada
arXiv AI
Oct 9

Scaling to Tens of Thousands of Test-Time Iterations with Loop-Native Attention Residuals

The paper introduces InfiLoop, a loop-native residual connection designed for looped Transformers to prevent performance degradation as iterations increase. InfiLoop learns which past computations to retain and how much to accept from each new update, combining content-based weighting with learned temporal decay to maintain a running summary of recurrent states. Across extensive reasoning tasks, a 7M-parameter InfiLoop model outperforms existing recursive architectures, achieving 97.9% exact accuracy on Sudoku-Extreme and 13.6% pass@2 on ARC-AGI-2, and continues to improve with test-time looping beyond 20,000 effective steps.

By Pengxiang Li, Dilxat Muhtar, Di He, Guinan Su, Lu Yin, Shiwei Liu
Hugging Face Trending Papers
Oct 8

Scaling to Tens of Thousands of Test-Time Iterations with Loop-Native Attention Residuals

The paper proposes InfiLoop, a loop-native residual connection designed for looped Transformers to prevent performance degradation as iterations increase. InfiLoop learns which past computations to retain and how much to accept from each new update, using content-based weighting and learned temporal decay to maintain a running summary of recurrent states. The approach yields strong results on reasoning tasks, achieving 97.9% exact accuracy on Sudoku-Extreme and 13.6% pass@2 on ARC-AGI-2, and continues to improve with over 20,000 test-time iterations.

arXiv Machine Learning
Aug 19

Recirculation

The paper introduces recirculation, an inference‑time architectural enhancement for foundation models that reduces perplexity and improves accuracy on generation and reasoning tasks without adding significant latency. Recirculation adds a specific form of recurrence, enabling the model to function as a dynamical system that tracks belief states, and is distinct from chain‑of‑thought or depth‑recurrence methods. An adaptive variant requires minimal hyperparameter tuning and achieves notable gains on the Gemma3 family, including a 23% perplexity drop and a 21% accuracy increase on GSM8k.

By Michael C. Mozer, Shoaib Ahmed Siddiqui, Danny Sawyer, Sunny Sanyal, Rosanne Liu
arXiv AI
Sep 3

CHASE: Cache-Hole-Adapted Skip Exit for Looped State-Space Language Models

The paper introduces CHASE, a cache‑hole‑adapted skip‑exit mechanism for looped state‑space language models, specifically Looped Mamba and Looped Hybrid Mamba‑Transformer. It shows that looping these architectures improves performance on controlled reasoning tasks and remains competitive in pre‑training benchmarks while using fewer distinct parameters. The cache‑hole adaptation allows selective skipping of recurrent steps during inference, maintaining perplexity close to full computation and achieving significant speedups.

By Zhenxuan Yu, Takeshi Kojima, Yutaka Matsuo, Yusuke Iwasawa
arXiv AI
Aug 26

Recursive Experiential-Working Memory Evolution for Long-Horizon Agent Harnesses

The paper introduces Recuris, a recursive Experiential‑Working Memory architecture that lets long‑horizon agents track task progress and select skills based on current needs rather than full history. By coupling working memory with experiential memory, execution becomes structured evidence that localizes failures to specific memory components, enabling a bounded recursive memory‑evolution loop. Across four benchmarks and ten models, Recuris improves task success in 35 of 37 model‑benchmark pairs, raising state‑of‑the‑art performance on tau‑bench and SkillFlow and reducing common long‑horizon failures by up to 80%.

By Zhaochen Yu, Yingcheng Wu, Zhenfei Yin, Kaiyuan Chen, Zhe Zhao, Mengdi Wang, Shuicheng Yan, Ling Yang
arXiv Machine Learning
Oct 28

Resample the Answer: Where to Spend Test-Time Compute in Recursive Reasoning

The paper investigates how to allocate test‑time computation in recursive reasoning models, specifically the Tiny Recursive Model (TRM). Experiments show that resampling the answer state (y) is more effective than perturbing the reasoning state (z) for recovering failures, especially as compute budgets increase. The proposed Gated Answer Resampling (GAR) method, which restarts only rejected puzzles, boosts TRM performance from 85.9% to 99.4% on Sudoku‑Extreme and from 68.8% to 85.6% on the Pencil Puzzle Bench, outperforming the probabilistic variant that uses far more compute.

By Maxime Illouz