arXiv AI

Stepwise Think-Critique: Interleaved Reasoning and Self-Critique in a Single LLM

The paper introduces Stepwise Think-Critique (STC), an end‑to‑end trainable framework that lets a single large language model (LLM) interleave reasoning with inline, step‑level critique. STC uses reinforcement learning to jointly reward reasoning accuracy and critique consistency, achieving a 7.2% improvement in Pass@1 on five mathematical reasoning benchmarks and a 67.4% step‑level critique F1 score, outperforming several external reward models. This approach aligns LLM behavior more closely with human critical thinking by embedding self‑evaluation directly into the reasoning process.

arXiv Machine Learning

Sep 4

Legibility is Not Interpretability: Comparing Judged and Actual Importance in Chain-Of-Thought Reasoning

The paper investigates whether the text of chain‑of‑thought reasoning steps actually reflects their true importance for a model’s final answer. By defining step importance as the advantage in expected reward when a step is included, the authors use Monte Carlo rollouts to estimate ground truth and then test whether large language model judges can identify high‑advantage steps. They find that capable LLMs can beat a prevalence baseline but still fall far short of a noise ceiling, and that fine‑tuning a step‑level critic improves detection for incorrect responses but remains distant from the ceiling for correct ones, indicating that step importance is only partially recoverable from the reasoning trace text.

By Kevin Du, Alexander Hoyle, Laura Ruis, Acyr Locatelli

llms reinforcement-learning fine-tuning safety

arXiv AI

Sep 16

Post-Training Large Language Models via Reinforcement Learning from Self-Feedback

arXiv:2507.21931v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) often produce plausible but poorly-calibrated answers, limiting their reliability on reasoning-intensive tasks....

By Carel van Niekerk, Renato Vukovic, Benjamin Ruppik, Hsien-chin Lin, Shutong Feng, Milica Ga\v{s}i\'c

llms nlp reinforcement-learning fine-tuning

arXiv AI

Jun 11

ProcessThinker: Enhancing Multi-modal Large Language Models Reasoning via Rollout-based Process Reward

arXiv:2606. 11209v1 Announce Type: cross Abstract: Visual question answering increasingly requires multi-step reasoning.

By Jingpei Wu, Xiao Han, Weixiang Shen, Boer Zhang, Zifeng Ding, Volker Tresp

llms nlp reinforcement-learning fine-tuning multimodal benchmarks

arXiv AI

Jun 2

ThinkSwitch: Context Distillation with LoRA and Weight Interpolation for Specific-Purpose Reasoning Tasks

arXiv:2606. 01080v1 Announce Type: cross Abstract: Large language models often improve on difficult tasks by spending inference-time compute on a reasoning trace before producing the final answer.

By Dhruv Saini, Rohan Pandey

llms fine-tuning efficiency

arXiv AI

Sep 21

LogicTrack: Auditing Reasoning Trajectories of Large Language Models with Formal Logic Solvers

arXiv:2609.21492v1 Announce Type: new Abstract: Chain-of-Thought (CoT) reasoning has been shown to improve the performance of large language models (LLMs), yet existing optimization methods largely r...

By Jingyu Hu, Shu Yang, Weiru Liu, Di Wang

llms fine-tuning benchmarks

arXiv AI

Jul 28

Offline-Online Curriculum RL for Multimodal Reasoning

arXiv:2607. 23700v1 Announce Type: new Abstract: Multimodal large language models exhibit capabilities on reasoning tasks, yet often produce flawed intermediate steps while yielding correct final answers.

By Wendi Deng, Hang Du, Guoshun Nan, Haokun Tian, Jiaqi Yu, Xinlei Cao, Jaile Li, Jingfeng Chen, Ling Deng, Ting Li, Hao Yang, Jun Liu, Xudong Jiang, Sicong Leng

llms reinforcement-learning multimodal benchmarks safety

arXiv AI

Jul 21

Can We Break LLMs Out of Self-Loops? Fine-Grained Reasoning Control with Activation Steering

arXiv:2607. 18100v1 Announce Type: new Abstract: Extended reasoning has become standard for frontier Large Language Models (LLMs), yet the trajectories these models produce remain largely uncontrollable.

By Sheldon Yu, Tong Yu, Xunyi Jiang, Rohan Surana, Gagan Mundada, Sungchul Kim, Lina Yao, Julian McAuley, Junda Wu

llms

arXiv Machine Learning

Sep 23

Ladders of Thought: A Self-Evolving Curriculum of Progressively Simplified Reasoning Traces

Ladders-of-Thought (LoT) is a framework that enhances reasoning in small- to mid-scale large language models by automatically generating easier variants of reasoning problems and organizing them into difficulty buckets. It uses a self‑evolving bandit scheduler to adaptively allocate training, improving performance across math and multi‑hop reasoning tasks on 1–8 B models. LoT achieves significant gains (e.g., +32 pp on AddSub, +16 pp on QASC) and converges faster than staged curricula.

By Minghui Liu, Thomas Magelinski, Dehao Yuan, Qi Yu, Furong Huang

llms reinforcement-learning efficiency

arXiv AI

Aug 12

ThinkRetrieve: Retrieval-Augmented Reasoning Traces for Test-Time Scaling

arXiv:2608. 10928v1 Announce Type: new Abstract: Large Reasoning Models (LRMs) improve performance by allocating additional inference-time compute to generate extended chain-of-thought reasoning.

By Vaibhav Singh, Soumya Suvra Ghosal, Sarvesh Gharat, Soumyabrata Pal, Ramasuri Narayanam, Dinesh Manocha

rag

arXiv AI

Aug 17

The Metacognitive Bottleneck: Japanese Riddles Reveal Fundamental Limits of Machine Insight and Self-Evaluation in Reasoning AI

arXiv:2509. 14704v3 Announce Type: replace Abstract: Benchmark saturation and training-data contamination increasingly obscure whether reported gains in large language models (LLMs) reflect genuine advances in reasoning or familiarity with recurring patterns in benchmark problems.

By Masaharu Mizumoto, Dat Nguyen, Zhiheng Han, Xingfu Li, Yo Nakawake, Le Minh Nguyen

llms benchmarks

arXiv AI

Sep 15

Post-Reasoning: Improving the Performance of Non-Thinking Models at No Cost

arXiv:2605.06165v2 Announce Type: replace Abstract: As the widespread adoption of Large Language Models (LLMs) accelerates, token consumption from intermediate reasoning traces increasingly contribut...

By Richmond Sin Jing Xuan, Rishabh Bhardwaj, Soujanya Poria

llms efficiency benchmarks

arXiv AI

3d ago

ArgGYM: A Procedural, Engine-Verified Benchmark for Structured Defeasible Reasoning

arXiv:2609.38409v1 Announce Type: new Abstract: Recent progress in large language model reasoning has been driven by benchmarks and reinforcement learning environments with automatically verifiable r...

By \.Ibrahim Ethem Deveci, Funda Tan \c{C}al{\i}k, Bar{\i}\c{s} Deniz Sa\u{g}lam, Duygu Ataman

llms reinforcement-learning benchmarks

Related stories