arXiv AI October 28, 2026 By Yueer Pan, Yuexi Pan

Correctness Conditioning as Selection in Chain-of-Thought Faithfulness Evaluation

The paper investigates how restricting chain‑of‑thought faithfulness evaluation to only correctly answered instances alters the measured population. Using a seven‑variable structural causal model, the authors derive an exact expression for the observed gap and distinguish between difficulty confounding and shortcut‑induced collider bias. A review of 18 papers and empirical evaluation on arithmetic tasks reveal positive selection gaps, with the GLM‑5.3 run showing a +7.4 percentage‑point increase and a pooled analysis yielding an odds ratio of 1.70.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Sep 24

Are Stated Reasoning Steps Causally Load-Bearing?

The study investigates whether the reasoning steps a language model writes are causally responsible for its answers. Using a causal intervention method on the activation stream, the authors find that for Qwen3-4B, about 77% of stated steps are causally load‑bearing, while behavioral tests overestimate this by roughly 11 percentage points. The faithfulness of reasoning decreases with model size and depth of reasoning, especially for the smaller Qwen3-1.7B.

By Abhiram Bhupatiraju, Rayan Nyaupane

arXiv AI

Jul 10

When LLMs Agree, Are They Right? Auditing Self-Consistency and Cross-Model Agreement as Confidence Signals

arXiv:2607. 08065v1 Announce Type: new Abstract: LLM-as-judge (Zheng et al.

By Kaihua Ding

llms safety

arXiv AI

Jun 15

C2-Faith: Benchmarking LLM Judges for Causal and Coverage Faithfulness in Chain-of-Thought Reasoning

arXiv:2603. 05167v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used as judges of chain-of-thought (CoT) reasoning, yet it remains unclear whether they can reliably assess process faithfulness rather than merely answer plausibility.

By Avni Mittal, Rauno Arike

llms benchmarks

arXiv AI

Jun 4

Reasoning or Fluency? Dissecting Probabilistic Confidence in Best-of-N Selection

arXiv:2601. 13735v2 Announce Type: replace Abstract: Probabilistic confidence metrics are increasingly adopted as proxies for reasoning quality in Best-of-N selection, under the assumption that higher confidence reflects higher reasoning fidelity.

By Hojin Kim, Jaehyung Kim

benchmarks

arXiv Machine Learning

Jun 16

Beyond Accuracy: Measuring Bias Acknowledgment in Chain-of-Thought Reasoning for Responsible AI Evaluation

arXiv:2606. 15127v1 Announce Type: new Abstract: Reasoning models are increasingly used in settings where the final answer is not the only object of review: educational tools may show students intermediate steps, decision-support systems may require human oversight, and audit workflows may inspect traces for misleading or biased input.

By Xian Sun, Wei Gao, Yingshuo Wang, Lingdong Kong, Yanhang Li, Zhichao Fan, Zexin Zhuang, Wenlong Dong, Zhiyuan Zheng, Hrishikesh Paranjape, Abhishek Mandal, Johnny R. Zhang

llms safety

arXiv AI

Aug 3

On the Generalization of Steering Vectors for Chain-of-Thought Faithfulness

arXiv:2607. 29062v1 Announce Type: new Abstract: Model capabilities have improved in large part due to scaling chain of thought.

By Matthew Nguyen, Kyle Cox, Austin Meek, Iv\'an Arcuschin

safety

One email a morning, machine-written

Related stories

Are Stated Reasoning Steps Causally Load-Bearing?

When LLMs Agree, Are They Right? Auditing Self-Consistency and Cross-Model Agreement as Confidence Signals

C2-Faith: Benchmarking LLM Judges for Causal and Coverage Faithfulness in Chain-of-Thought Reasoning

Reasoning or Fluency? Dissecting Probabilistic Confidence in Best-of-N Selection

Beyond Accuracy: Measuring Bias Acknowledgment in Chain-of-Thought Reasoning for Responsible AI Evaluation

On the Generalization of Steering Vectors for Chain-of-Thought Faithfulness