arXiv Computation and Language
LoopRPT: Reinforcement Pre-Training for Looped Language Models
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
Jun 6
CoT-Space: A Theoretical Framework for Internal Slow-Thinking via Reinforcement Learning
arXiv:2509. 04027v3 Announce Type: replace Abstract: Test-time scaling, primarily manifested through multi-step Chain-of-Thought (CoT) reasoning via Reinforcement Learning (RL), has emerged as a pivotal paradigm for enhancing the reasoning capabilities of Large Language Models (LLMs).
Hugging Face Trending Papers
Oct 7
Recurrent Self-Improvement: Dynamic Cross-Loop On-Policy Distillation for Looped Language Models
Looped Language Models (LoopLMs) offer a parameter efficient approach to scaling reasoning by reusing shared parameters across recurrent computation steps. Despite their promise, effective post-traini...
arXiv AI
Oct 9
From Chain-of-Thought to Loops: Non-Autoregressive Latent Reasoning via Looped Transformers
The paper introduces LLoCoT, a looped latent‑reasoning framework that replaces the traditional left‑to‑right generation of chain‑of‑thought tokens with iterative refinement of a compact latent workspace using a shared transformer. By jointly updating latent slots over a few iterations and sampling latent tokens in parallel, LLoCoT feeds these into an autoregressive decoder for answer generation. Experiments on HumanEval and MBPP show that LLoCoT matches or surpasses explicit‑CoT baselines while dramatically reducing reasoning latency and increasing throughput.
arXiv AI
Oct 2
Closing the Loop: Practical Training Recipes for Looped Language Models
The paper presents practical training recipes for looped language models, showing that a compute‑efficient pipeline can reduce the training budget from 7.7 T tokens to 310 B tokens while maintaining strong reasoning performance. In controlled experiments, a 1.4 B LoopLM outperforms a parameter‑matched dense model on 12 benchmarks, achieving significant gains on GSM8K, MATH, and DROP, and approaches a 3.9 B dense model at matched inference compute. The authors also provide a minimal recipe to convert pretrained dense models into looped ones, demonstrating improvements on Qwen3‑1.7B‑Base across multiple data regimes.
Hugging Face Trending Papers
Jun 22
SPIRAL: Learning to Search and Aggregate
Language model reasoning can be substantially improved at test time via scaffolds that scale inference compute across different primitives -- sequential reasoning within a trace, independently sampled parallel traces, and aggregation of multiple reasoning traces into a final response. During post-training, however, language models are optimized only for sequential reasoning within a single trace.
arXiv AI
Oct 2
SPIRAL: Learning to Search and Aggregate
SPIRAL is a reinforcement‑learning framework that trains language models to employ three inference primitives—sequential reasoning within a trace, parallel sampling of independent traces, and aggregation of those traces—within a single compute pipeline. The model first generates multiple independent chain‑of‑thought traces in parallel, then produces a final aggregation trace conditioned on them, with all components optimized end‑to‑end for the reward of the aggregated response. Experiments on reasoning tasks demonstrate that SPIRAL scales efficiently with inference compute, achieving up to 11× better scaling efficiency and 15% higher performance compared to the GRPO baseline when all three primitives are scaled.