← Back to all news
arXiv Machine Learning By Xin Xu, Clive Bai, Kai Yang, Tianhao Chen, Yang Wang, Saiyong Yang, Can Yang

Thinking-Free Policy Initialization Makes Distilled Reasoning Models More Effective and Efficient Reasoners

Read the original on arXiv Machine Learning →

The paper introduces Thinking-Free Policy Initialization (TFPI), a simple adaptation to Reinforcement Learning with Verifiable Reward (RLVR) that removes explicit thinking content during training and inference. TFPI bridges long Chain-of-Thought distillation with standard RLVR, reducing token usage and improving performance. Experiments show that TFPI accelerates RL convergence, raises performance ceilings, and yields token‑efficient reasoning models, enabling a 4B model to achieve 89.0% accuracy on AIME24 and 65.5% on LiveCodeBench with under 4K H20 hours.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
5d ago

Efficiently Learning To Reason or Not to Reason: Root-token Policy Optimization for Adaptive Thinking

The paper introduces Root-token Policy Optimization (RPO), a framework that lets large reasoning models decide when to engage in explicit reasoning by training only the initial root token. RPO uses group-relative rewards and advantages to focus on this branching point, dramatically cutting training overhead and VRAM usage. Experiments across various model families show that RPO achieves difficulty-aware adaptive thinking with roughly 2% of the compute required by previous adaptive-reasoning methods.

By Taehyeon Kim, Hyunsoo Lee, Youngsoo Jang, Moontae Lee
More like this →
arXiv Computation and Language
5d ago

Draft-Thinking: Learning Efficient Reasoning in Long Chain-of-Thought LLMs

arXiv:2603.00578v2 Announce Type: replace-cross Abstract: Long chain-of-thought~(CoT) reasoning improves the problem-solving capability of large reasoning models~(LRMs), but often incurs substantial...

By Jie Cao, Tianwei Lin, Juekai Lin, Zhenxuan Fan, Yunqi Cao, Bo Yuan, Ziyuan Zhao, Rolan Yan, Wenqiao Zhang, Siliang Tang
llmsreinforcement-learningfine-tuningbenchmarks
More like this →
arXiv AI
Sep 7

Extremely Sparse Supervision Incentivizes Reasoning Ability

The paper reports that in on‑policy distillation for large language models, reasoning performance can be improved by supervising only a tiny fraction of generated tokens—sometimes just one or two tokens per reasoning trajectory, about 0.05% of all tokens. This sparse supervision consistently matches or exceeds full‑token training across nine teacher‑student setups on mathematical reasoning, and is also validated on coding reasoning, Llama models, and PPO‑based reinforcement learning with verifiable reward. The findings suggest that effective post‑training does not require token‑intensive supervision and may align more closely with natural learning processes that focus on critical reasoning steps.

By Zhishuai Liu, Xingzi Xu, Mehmet Saygin Seyfioglu, Pan Xu, Karim Bouyarmane
llmsreinforcement-learningefficiency
More like this →