arXiv AI September 3, 2026 By Jiaqi Xu, Cuiling Lan, Xuejin Chen, Yan Lu

Stepwise Think-Critique: Interleaved Reasoning and Self-Critique in a Single LLM

The paper introduces Stepwise Think-Critique (STC), an end‑to‑end trainable framework that lets a single large language model (LLM) interleave reasoning with inline, step‑level critique. STC uses reinforcement learning to jointly reward reasoning accuracy and critique consistency, achieving a 7.2% improvement in Pass@1 on five mathematical reasoning benchmarks and a 67.4% step‑level critique F1 score, outperforming several external reward models. This approach aligns LLM behavior more closely with human critical thinking by embedding self‑evaluation directly into the reasoning process.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning

Sep 4

Legibility is Not Interpretability: Comparing Judged and Actual Importance in Chain-Of-Thought Reasoning

The paper investigates whether the text of chain‑of‑thought reasoning steps actually reflects their true importance for a model’s final answer. By defining step importance as the advantage in expected reward when a step is included, the authors use Monte Carlo rollouts to estimate ground truth and then test whether large language model judges can identify high‑advantage steps. They find that capable LLMs can beat a prevalence baseline but still fall far short of a noise ceiling, and that fine‑tuning a step‑level critic improves detection for incorrect responses but remains distant from the ceiling for correct ones, indicating that step importance is only partially recoverable from the reasoning trace text.

By Kevin Du, Alexander Hoyle, Laura Ruis, Acyr Locatelli

llms reinforcement-learning fine-tuning safety

arXiv AI

Sep 16

Post-Training Large Language Models via Reinforcement Learning from Self-Feedback

arXiv:2507.21931v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) often produce plausible but poorly-calibrated answers, limiting their reliability on reasoning-intensive tasks....

By Carel van Niekerk, Renato Vukovic, Benjamin Ruppik, Hsien-chin Lin, Shutong Feng, Milica Ga\v{s}i\'c

llms nlp reinforcement-learning fine-tuning

arXiv AI

Jun 11

ProcessThinker: Enhancing Multi-modal Large Language Models Reasoning via Rollout-based Process Reward

arXiv:2606. 11209v1 Announce Type: cross Abstract: Visual question answering increasingly requires multi-step reasoning.

By Jingpei Wu, Xiao Han, Weixiang Shen, Boer Zhang, Zifeng Ding, Volker Tresp

llms nlp reinforcement-learning fine-tuning multimodal benchmarks

arXiv AI

Jun 2

ThinkSwitch: Context Distillation with LoRA and Weight Interpolation for Specific-Purpose Reasoning Tasks

arXiv:2606. 01080v1 Announce Type: cross Abstract: Large language models often improve on difficult tasks by spending inference-time compute on a reasoning trace before producing the final answer.

By Dhruv Saini, Rohan Pandey

llms fine-tuning efficiency

arXiv AI

Sep 21

LogicTrack: Auditing Reasoning Trajectories of Large Language Models with Formal Logic Solvers

arXiv:2609.21492v1 Announce Type: new Abstract: Chain-of-Thought (CoT) reasoning has been shown to improve the performance of large language models (LLMs), yet existing optimization methods largely r...

By Jingyu Hu, Shu Yang, Weiru Liu, Di Wang

llms fine-tuning benchmarks

arXiv AI

Jul 28

Offline-Online Curriculum RL for Multimodal Reasoning

arXiv:2607. 23700v1 Announce Type: new Abstract: Multimodal large language models exhibit capabilities on reasoning tasks, yet often produce flawed intermediate steps while yielding correct final answers.

By Wendi Deng, Hang Du, Guoshun Nan, Haokun Tian, Jiaqi Yu, Xinlei Cao, Jaile Li, Jingfeng Chen, Ling Deng, Ting Li, Hao Yang, Jun Liu, Xudong Jiang, Sicong Leng

llms reinforcement-learning multimodal benchmarks safety

One email a morning, machine-written

Related stories

Legibility is Not Interpretability: Comparing Judged and Actual Importance in Chain-Of-Thought Reasoning

Post-Training Large Language Models via Reinforcement Learning from Self-Feedback

ProcessThinker: Enhancing Multi-modal Large Language Models Reasoning via Rollout-based Process Reward

ThinkSwitch: Context Distillation with LoRA and Weight Interpolation for Specific-Purpose Reasoning Tasks

LogicTrack: Auditing Reasoning Trajectories of Large Language Models with Formal Logic Solvers

Offline-Online Curriculum RL for Multimodal Reasoning