The paper introduces Elastic Latent Recurrence (ELR), a training recipe that enables looped language models to match the accuracy of non-recurrent models at a single loop while allowing additional loops to improve accuracy with more compute. ELR achieves this through gated state updates with learned step sizes, mixed loop counts during training, and conditioning on loop index and budget. The method also reduces training and inference overhead in Mixture-of-Experts models and produces a single checkpoint, Koi‑4B, that performs competitively with state‑of‑the‑art models on multiple benchmarks.
By Faisal Ladhak, Shantanu Acharya, Santiago Akle Serano, Simeng Sun, Boris Ginsburg, Krishna C Puvvada
arXiv:2609.36636v1 Announce Type: new
Abstract: Looped language models (LoopLMs) increase computational depth through parameter sharing, offering a path to scale inference computation without adding...
By Xinlin Zhuang, Siyuan Wang, Imran Razzak, Weiyang Liu
The paper introduces InfiLoop, a loop-native residual connection designed for looped Transformers to prevent performance degradation as iterations increase. InfiLoop learns which past computations to retain and how much to accept from each new update, combining content-based weighting with learned temporal decay to maintain a running summary of recurrent states. Across extensive reasoning tasks, a 7M-parameter InfiLoop model outperforms existing recursive architectures, achieving 97.9% exact accuracy on Sudoku-Extreme and 13.6% pass@2 on ARC-AGI-2, and continues to improve with test-time looping beyond 20,000 effective steps.
By Pengxiang Li, Dilxat Muhtar, Di He, Guinan Su, Lu Yin, Shiwei Liu
The paper proposes InfiLoop, a loop-native residual connection designed for looped Transformers to prevent performance degradation as iterations increase. InfiLoop learns which past computations to retain and how much to accept from each new update, using content-based weighting and learned temporal decay to maintain a running summary of recurrent states. The approach yields strong results on reasoning tasks, achieving 97.9% exact accuracy on Sudoku-Extreme and 13.6% pass@2 on ARC-AGI-2, and continues to improve with over 20,000 test-time iterations.
arXiv:2511.08577v4 Announce Type: replace-cross
Abstract: Improving the reasoning abilities of Large Language Models (LLMs), especially under parameter constraints, is crucial for real-world applicat...
By Tianyu Fu, Yichen You, Zekai Chen, Guohao Dai, Huazhong Yang, Yu Wang
The paper introduces recirculation, an inference‑time architectural enhancement for foundation models that reduces perplexity and improves accuracy on generation and reasoning tasks without adding significant latency. Recirculation adds a specific form of recurrence, enabling the model to function as a dynamical system that tracks belief states, and is distinct from chain‑of‑thought or depth‑recurrence methods. An adaptive variant requires minimal hyperparameter tuning and achieves notable gains on the Gemma3 family, including a 23% perplexity drop and a 21% accuracy increase on GSM8k.
By Michael C. Mozer, Shoaib Ahmed Siddiqui, Danny Sawyer, Sunny Sanyal, Rosanne Liu