arXiv AI

Counterfactual Simulation Training for Chain-of-Thought Faithfulness

arXiv:2602. 20710v2 Announce Type: replace Abstract: Inspecting Chain-of-Thought reasoning is among the most common means of understanding why an LLM produced its output.

arXiv AI

Jun 26

Where Do CoT Training Gains Land in LLM based Agents?

arXiv:2606. 26935v1 Announce Type: new Abstract: Chain-of-thought (CoT) reasoning is widely used in language-model agents, but prior work has shown that verbalized CoT is not always faithful and may instead reflect post-hoc reasoning, which means the model already knows the answer before reasoning.

By Jingyu Liu, Zhiwen Wang, Yuxin Jing, Huanyu Zhou, Yong Liu

llms agents

arXiv AI

Aug 3

On the Generalization of Steering Vectors for Chain-of-Thought Faithfulness

arXiv:2607. 29062v1 Announce Type: new Abstract: Model capabilities have improved in large part due to scaling chain of thought.

By Matthew Nguyen, Kyle Cox, Austin Meek, Iv\'an Arcuschin

safety

arXiv AI

Sep 1

Chain-of-Thought Faithfulness of Reasoning Models Varies with Where and How Preference Cues Are Delivered

arXiv:2608.29464v1 Announce Type: cross Abstract: Chain-of-thought (CoT) monitoring assumes that reasoning traces faithfully record the information that shapes a model's answer. Existing faithfulness...

By Aryo Pradipta Gema, Neel Rajani, Rohit Saxena, Wai-Chung Kwan, Pasquale Minervini

llms agents safety

arXiv AI

Jun 11

Calibration Drift Under Reasoning: How Chain-of-Thought Budgets Induce Overconfidence in Large Language Models

arXiv:2606. 11211v1 Announce Type: cross Abstract: The ability of large language models (LLMs) to express calibrated uncertainty is important for safe deployment.

By Prakul Sunil Hiremath, Harshit R. Hiremath

llms

arXiv AI

Jul 2

Verbosity Tradeoffs and the Impact of Scale on the Faithfulness of LLM Self-Explanations

arXiv:2503. 13445v3 Announce Type: replace-cross Abstract: When asked to explain their decisions, LLMs can often give explanations which sound plausible to humans.

By Noah Y. Siegel, Nicolas Heess, Maria Perez-Ortiz, Oana-Maria Camburu

llms

arXiv AI

Aug 11

Understanding and Mitigating Premature Confidence for Better LLM Reasoning

arXiv:2605. 24396v2 Announce Type: replace Abstract: Long chains of thought (CoT) from current language models frequently contain logical gaps and unjustified leaps, limiting the gains from additional test-time compute.

By Jingchu Gai, Guanning Zeng, Christina Baek, Chen Wu, J. Zico Kolter, Andrej Risteski, Aditi Raghunathan

llms reinforcement-learning benchmarks safety

arXiv AI

Jul 24

Training Large Language Models for Self-Explanation Faithfulness

arXiv:2607. 21090v1 Announce Type: cross Abstract: We propose a Reinforcement Learning (RL) method to directly optimize the faithfulness of self-explanations - the extent to which a model's generated reasoning accurately reflects its internal decision-making process.

By Yeoktatt Cheah, Mar\'ia P\'erez-Ortiz, Noah Y. Siegel, Oana-Maria Camburu

llms reinforcement-learning fine-tuning safety

arXiv Machine Learning

Jun 2

Enhancing LLM Metacognition via Cognitive Pairwise Training

arXiv:2606. 00869v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has become central to LLM reasoning, but its outcome-level rewards can make models more willing to give confident answers when evidence or reasoning is unreliable.

By Weitao Li, Hao Zhou, Xuanyu Lei, Fandong Meng, Yuanhang Liu, Jingyi Ren, Ante Wang, Xiaolong Wang, Yuanchi Zhang, Fuwen Luo, Guangwen Yang, Lin Gan, Weizhi Ma, Yang Liu

llms reinforcement-learning fine-tuning safety

arXiv AI

3d ago

Beyond the Shadows of Plato's Cave: Evaluating False Memory in Autonomous Agents via Counterfactual Reasoning

The paper introduces FAME, a training‑free framework that evaluates false memory in autonomous agents by tracking how their internal beliefs shift under counterfactual scenarios. False memory, defined as biases arising from spurious correlations, environment shifts, or knowledge conflicts, is hard to detect with standard methods. FAME measures concept drift in hidden states, achieving AUROCs between 76.2% and 96.7% and outperforming baselines on benchmarks such as GSM‑Symbolic, GitChameleon, and BigBench‑Hard.

By Quan M. Tran, Zhuo Huang, Zhen Fang, Jing Zhang, Mingming Gong, Tongliang Liu

agents benchmarks

arXiv AI

2d ago

On Language Drift during RLVR Post-Training

arXiv:2610.02015v1 Announce Type: cross Abstract: Recent advances in LLM reasoning models---driven primarily by the paradigm of post-training via reinforcement learning with verifiable reward (RLVR)-...

By Michael Sullivan, Alexander Koller

llms reinforcement-learning fine-tuning

arXiv AI

Sep 1

Drop the Act: Probe-Filtered RL for Faithful Chain-of-Thought Reasoning

arXiv:2605.11467v2 Announce Type: replace-cross Abstract: Reasoning models post-hoc rationalize answers they have already committed to internally, producing chains of *reasoning theater*: deliberativ...

By Swapnil Parekh, Naman Goyal

llms reinforcement-learning benchmarks safety

arXiv AI

Aug 5

The Tell-Tale Trace: Detecting Reasoning Failures in LLMs Using Chain-of-Thought Dynamics

arXiv:2608. 03291v1 Announce Type: cross Abstract: Chain-of-thought (CoT) reasoning improves large language model (LLM) performance while also providing an observable interface to the model's reasoning process.

By Shashwat Sourav, Aishwarya Balwani

llms

Related stories