arXiv Machine Learning
What Makes Recurrence Effective in Looped Language Models?
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Sep 3
CHASE: Cache-Hole-Adapted Skip Exit for Looped State-Space Language Models
The paper introduces CHASE, a cache‑hole‑adapted skip‑exit mechanism for looped state‑space language models, specifically Looped Mamba and Looped Hybrid Mamba‑Transformer. It shows that looping these architectures improves performance on controlled reasoning tasks and remains competitive in pre‑training benchmarks while using fewer distinct parameters. The cache‑hole adaptation allows selective skipping of recurrent steps during inference, maintaining perplexity close to full computation and achieving significant speedups.
arXiv AI
Jul 14
Looped State-Space Language Models with Adaptive Exit-State Selection
arXiv:2607. 10110v1 Announce Type: new Abstract: Recent work on looped language models suggests that many reasoning problems benefit from greater computational depth rather than from additional independent parameters.
arXiv Machine Learning
Aug 19
Recirculation
The paper introduces recirculation, an inference‑time architectural enhancement for foundation models that reduces perplexity and improves accuracy on generation and reasoning tasks without adding significant latency. Recirculation adds a specific form of recurrence, enabling the model to function as a dynamical system that tracks belief states, and is distinct from chain‑of‑thought or depth‑recurrence methods. An adaptive variant requires minimal hyperparameter tuning and achieves notable gains on the Gemma3 family, including a 23% perplexity drop and a 21% accuracy increase on GSM8k.
arXiv AI
Aug 20
Looped Language Models Improve Compositional Tool Calling
The paper investigates the use of looped language models for compositional tool calling, where models must coordinate multiple API calls and maintain state across interactions. Experiments on API-Bank, BFCL, and NESTful show that recurrent computation generally improves compositional and dependency-aware tool use, with accuracy increasing as recurrent depth grows. Adaptive inference offers a better compute‑performance trade‑off by allocating extra computation only when necessary.
arXiv Machine Learning
Sep 4
RecurTrace: Adaptive Latent Reasoning with Loop-Time Memory
RecurTrace introduces adaptive latent reasoning for language models by allowing each looped layer to attend to its own past states and by using a halting head to decide when to stop iterating. This approach overcomes two limitations of prior latent recurrence methods: limited access to earlier computations and a fixed loop count that mismatches input difficulty. In experiments on MathQA, RecurTrace achieves 56.9% accuracy with an average of 2.0 loops, outperforming fixed‑depth baselines and other adaptive methods, and it also improves generation accuracy across a range of model sizes.
arXiv AI
Aug 12
Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformers
arXiv:2604. 07822v2 Announce Type: replace-cross Abstract: We study implicit reasoning, i.