arXiv AI
Sep 7

Extremely Sparse Supervision Incentivizes Reasoning Ability

The paper reports that in on‑policy distillation for large language models, reasoning performance can be improved by supervising only a tiny fraction of generated tokens—sometimes just one or two tokens per reasoning trajectory, about 0.05% of all tokens. This sparse supervision consistently matches or exceeds full‑token training across nine teacher‑student setups on mathematical reasoning, and is also validated on coding reasoning, Llama models, and PPO‑based reinforcement learning with verifiable reward. The findings suggest that effective post‑training does not require token‑intensive supervision and may align more closely with natural learning processes that focus on critical reasoning steps.

By Zhishuai Liu, Xingzi Xu, Mehmet Saygin Seyfioglu, Pan Xu, Karim Bouyarmane
arXiv Computation and Language
Oct 23

Mimir: A Sub-2B Hierarchical Reasoning Model Trained on Permissible Data Punches Above Its Weight Class

Mimir is a 1.78‑B parameter language model built on the Hierarchical Reasoning Model Text (HRM‑Text) architecture, which interleaves low‑level and high‑level transformer stacks. It is trained from scratch on a curated mix of 161 instruction, reasoning, tool‑use, multilingual, and synthetic datasets totaling 70.5 B tokens per epoch (103 B for the continued‑training version Mimir v1.5), using only data that meets permissibility criteria. Across 20 benchmarks in English, Danish, and Math & Code, Mimir v1.5 outperforms all other sub‑2 B models and matches or exceeds the performance of larger 9 B‑parameter models, while memorisation audits show low risk and inference‑time ablations reveal graceful degradation when adjusting low‑level stack applications.

By Peter Schneider-Kamp, Jacob Nielsen, Gianluca Barmina, Danial Namazifard, Annemette Brok Pirchert, Kenneth Enevoldsen, Lukas Galke Poech
arXiv AI
Oct 2

Closing the Loop: Practical Training Recipes for Looped Language Models

The paper presents practical training recipes for looped language models, showing that a compute‑efficient pipeline can reduce the training budget from 7.7 T tokens to 310 B tokens while maintaining strong reasoning performance. In controlled experiments, a 1.4 B LoopLM outperforms a parameter‑matched dense model on 12 benchmarks, achieving significant gains on GSM8K, MATH, and DROP, and approaches a 3.9 B dense model at matched inference compute. The authors also provide a minimal recipe to convert pretrained dense models into looped ones, demonstrating improvements on Qwen3‑1.7B‑Base across multiple data regimes.

By Andrei Marchenko, Viacheslav Bezrukov, Oleg Kashurin, Inessa Fedorova, Dmitry Bocharov, Yuliana Shakhvalieva, Maria Tikhonova, Valerii Ternovskii
arXiv AI
Oct 2

Masked Self-Distillation: Internalizing the Chain-of-Thought in Language Models

The paper introduces masked self‑distillation, a post‑training framework that trains a language model to internalize portions of its own intermediate reasoning traces. By varying the fraction of trace internalized, the authors demonstrate that models can achieve higher inference efficiency and improved task performance on math and graph‑coloring problems. Experiments on Qwen3‑4B and Qwen3‑8B show that the method generalizes well to in‑domain out‑of‑distribution cases without catastrophic forgetting, and that supervised fine‑tuning alone can reduce trace length at the expense of generalization.

By Durgesh Kalwar, Vardhan Palod, Jaya Adithya Pavuluri, Subbarao Kambhampati
arXiv AI
Jul 28

DeepLook: Deeper Thinking with Lookahead

arXiv:2607. 22602v1 Announce Type: new Abstract: Inference-time scaling has emerged as a powerful paradigm for improving large language model reasoning, often delivering larger gains on difficult reasoning tasks than parameter scaling alone.

By Tingxin Yang, Zefeng Wang, Mengyue Wang, Xingcheng Zhou, Yunpu Ma