The paper introduces CHASE, a cache‑hole‑adapted skip‑exit mechanism for looped state‑space language models, specifically Looped Mamba and Looped Hybrid Mamba‑Transformer. It shows that looping these architectures improves performance on controlled reasoning tasks and remains competitive in pre‑training benchmarks while using fewer distinct parameters. The cache‑hole adaptation allows selective skipping of recurrent steps during inference, maintaining perplexity close to full computation and achieving significant speedups.
By Zhenxuan Yu, Takeshi Kojima, Yutaka Matsuo, Yusuke Iwasawa
The paper presents practical training recipes for looped language models, showing that a compute‑efficient pipeline can reduce the training budget from 7.7 T tokens to 310 B tokens while maintaining strong reasoning performance. In controlled experiments, a 1.4 B LoopLM outperforms a parameter‑matched dense model on 12 benchmarks, achieving significant gains on GSM8K, MATH, and DROP, and approaches a 3.9 B dense model at matched inference compute. The authors also provide a minimal recipe to convert pretrained dense models into looped ones, demonstrating improvements on Qwen3‑1.7B‑Base across multiple data regimes.
By Andrei Marchenko, Viacheslav Bezrukov, Oleg Kashurin, Inessa Fedorova, Dmitry Bocharov, Yuliana Shakhvalieva, Maria Tikhonova, Valerii Ternovskii
The paper introduces InfiLoop, a loop-native residual connection designed for looped Transformers to prevent performance degradation as iterations increase. InfiLoop learns which past computations to retain and how much to accept from each new update, combining content-based weighting with learned temporal decay to maintain a running summary of recurrent states. Across extensive reasoning tasks, a 7M-parameter InfiLoop model outperforms existing recursive architectures, achieving 97.9% exact accuracy on Sudoku-Extreme and 13.6% pass@2 on ARC-AGI-2, and continues to improve with test-time looping beyond 20,000 effective steps.
By Pengxiang Li, Dilxat Muhtar, Di He, Guinan Su, Lu Yin, Shiwei Liu
arXiv:2607. 10110v1 Announce Type: new Abstract: Recent work on looped language models suggests that many reasoning problems benefit from greater computational depth rather than from additional independent parameters.
By Zhenxuan Yu, Takeshi Kojima, Yutaka Matsuo, Yusuke Iwasawa
Looped Language Models (LoopLMs) offer a parameter efficient approach to scaling reasoning by reusing shared parameters across recurrent computation steps. Despite their promise, effective post-traini...
The paper introduces recirculation, an inference‑time architectural enhancement for foundation models that reduces perplexity and improves accuracy on generation and reasoning tasks without adding significant latency. Recirculation adds a specific form of recurrence, enabling the model to function as a dynamical system that tracks belief states, and is distinct from chain‑of‑thought or depth‑recurrence methods. An adaptive variant requires minimal hyperparameter tuning and achieves notable gains on the Gemma3 family, including a 23% perplexity drop and a 21% accuracy increase on GSM8k.
By Michael C. Mozer, Shoaib Ahmed Siddiqui, Danny Sawyer, Sunny Sanyal, Rosanne Liu
The paper investigates the use of looped language models for compositional tool calling, where models must coordinate multiple API calls and maintain state across interactions. Experiments on API-Bank, BFCL, and NESTful show that recurrent computation generally improves compositional and dependency-aware tool use, with accuracy increasing as recurrent depth grows. Adaptive inference offers a better compute‑performance trade‑off by allocating extra computation only when necessary.
By Andrei Cristian Popescu, Haitz S\'aez de Oc\'ariz Borde, Pietro Li\`o
RecurTrace introduces adaptive latent reasoning for language models by allowing each looped layer to attend to its own past states and by using a halting head to decide when to stop iterating. This approach overcomes two limitations of prior latent recurrence methods: limited access to earlier computations and a fixed loop count that mismatches input difficulty. In experiments on MathQA, RecurTrace achieves 56.9% accuracy with an average of 2.0 loops, outperforming fixed‑depth baselines and other adaptive methods, and it also improves generation accuracy across a range of model sizes.
By Yuxiang Wang, Kunyu Feng, Yingda Shen, Haoning Xu, Junyu Wang, Zhizheng Wu
arXiv:2604. 07822v2 Announce Type: replace-cross Abstract: We study implicit reasoning, i.
By Harsh Kohli, Srinivasan Parthasarathy, Huan Sun, Yuekun Yao
The paper proposes InfiLoop, a loop-native residual connection designed for looped Transformers to prevent performance degradation as iterations increase. InfiLoop learns which past computations to retain and how much to accept from each new update, using content-based weighting and learned temporal decay to maintain a running summary of recurrent states. The approach yields strong results on reasoning tasks, achieving 97.9% exact accuracy on Sudoku-Extreme and 13.6% pass@2 on ARC-AGI-2, and continues to improve with over 20,000 test-time iterations.
LoopOPD introduces a cross-loop on-policy distillation framework that uses a frozen terminal loop policy as a compute-privileged teacher for intermediate loop students, providing dense supervision without external teachers. Dynamic LoopOPD (D-LoopOPD) further refreshes the terminal loop teacher as model parameters update, enabling recurrent self‑improvement. Experiments on Ouro‑Thinking models show that LoopOPD enhances mathematical reasoning and that D‑LoopOPD yields additional gains, with the models also improving on general reasoning and code generation benchmarks.
By Yi Wang, Rui Qian, Yu Li, Haoyang Yao, Wenjie Wang
LoopLook introduces a lightweight encoder-side recurrent perception framework that equips pretrained Vision‑Language Models with adaptive visual depth. By repeatedly revisiting a middle‑to‑late window of the vision encoder and injecting question representations at each loop, it refines visual evidence in a task‑directed manner. The learned stopping mechanism and token‑wise Mixture‑of‑Looks allow each visual token to selectively integrate information across loops, yielding consistent performance gains across multiple backbones and benchmarks.
By Wencheng Ye, Anning Hu, Hongye Fang, Tianyi Wang, Junchi Yan