← Back to all news
arXiv Machine Learning By Eric O. Korman

Inverse Reasoning Models for Efficient Post-Training and Verification from Answer-Only Data

Read the original on arXiv Machine Learning →

The paper introduces a self‑distillation method that leverages answer‑only data to improve large language models (LLMs) after training. By training an inverse reasoning LLM to generate latent chain‑of‑thought traces conditioned on the answer, the authors can perform offline distillation that significantly boosts performance on hard prompts for tasks like Countdown and GSM8K. Additionally, the inverse model serves as an accurate verifier, enhancing best‑of‑n selection accuracy.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Sep 1

Locked at the Entrance, Open Inside: Where RLVR Narrows the Solution Space

The paper investigates why reinforcement learning with verifiable rewards (RLVR) reduces the diversity of solutions in reasoning tasks. By analyzing the Countdown task, the authors show that RLVR contracts the solution space mainly at the entrance—before the first arithmetic operation—causing a 67% drop in solution coverage. They demonstrate that providing an unselected entrance prefix or applying entrance‑targeted interventions can restore or even improve coverage without harming accuracy.

By Qiancheng Zhou, Ruizhe Li
llmsreinforcement-learningfine-tuningbenchmarks
More like this →
arXiv AI
Aug 7

DASH: Divergence-Adaptive Supervision Horizons for On-Policy Self-Distillation of Reasoning Models

arXiv:2608. 06243v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) improves the reasoning capabilities of large language models using automatically verifiable outcome signals, but these signals are typically sparse and at the sequence-level.

By ZhiYan Hou, Xinyu Tang, Hongyan An, Jianjin Zhang, Weizhen Wang, Yunyun Han, Gengsheng Li, Xiangzhao Hao, Haiyun Guo, Wenbin Hu, Jinqiao Wang, Yafeng Deng
llmsreinforcement-learningefficiencybenchmarks
More like this →
arXiv AI
Oct 2

Masked Self-Distillation: Internalizing the Chain-of-Thought in Language Models

The paper introduces masked self‑distillation, a post‑training framework that trains a language model to internalize portions of its own intermediate reasoning traces. By varying the fraction of trace internalized, the authors demonstrate that models can achieve higher inference efficiency and improved task performance on math and graph‑coloring problems. Experiments on Qwen3‑4B and Qwen3‑8B show that the method generalizes well to in‑domain out‑of‑distribution cases without catastrophic forgetting, and that supervised fine‑tuning alone can reduce trace length at the expense of generalization.

By Durgesh Kalwar, Vardhan Palod, Jaya Adithya Pavuluri, Subbarao Kambhampati
llmsfine-tuningefficiencybenchmarks
More like this →