arXiv Machine Learning June 16, 2026 By Xian Sun, Wei Gao, Yingshuo Wang, Lingdong Kong, Yanhang Li, Zhichao Fan, Zexin Zhuang, Wenlong Dong, Zhiyuan Zheng, Hrishikesh Paranjape, Abhishek Mandal, Johnny R. Zhang

Beyond Accuracy: Measuring Bias Acknowledgment in Chain-of-Thought Reasoning for Responsible AI Evaluation

Read the original on arXiv Machine Learning →

arXiv:2606. 15127v1 Announce Type: new Abstract: Reasoning models are increasingly used in settings where the final answer is not the only object of review: educational tools may show students intermediate steps, decision-support systems may require human oversight, and audit workflows may inspect traces for misleading or biased input.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 9

Reasoning Consistency Scanning: A Framework for Auditing Chain-of-Thought Validity in AI Safety Evaluations

arXiv:2607. 07229v1 Announce Type: new Abstract: Prior work has shown that chain-of-thought (CoT) reasoning is often unfaithful: a model's stated reasoning does not reliably reflect the process that produced its output.

By Silvia Santano

benchmarks safety

arXiv AI

Jul 31

A Reference-Free Score for Detecting Silent Reasoning Failures in Large Language Models

arXiv:2607. 26102v1 Announce Type: cross Abstract: Mathematical chain of thought (CoT) evaluation is commonly reduced to whether the final answer matches a reference.

By Vivek Shukla, Varun Shukla, Atul, Divya Mishra, Mehul Kumar Das

llms

arXiv AI

Jun 15

C2-Faith: Benchmarking LLM Judges for Causal and Coverage Faithfulness in Chain-of-Thought Reasoning

arXiv:2603. 05167v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used as judges of chain-of-thought (CoT) reasoning, yet it remains unclear whether they can reliably assess process faithfulness rather than merely answer plausibility.

By Avni Mittal, Rauno Arike

llms benchmarks

arXiv AI

Jun 24

Grading the Grader: Lessons from Evaluating an Agentic Data Analysis System

arXiv:2606. 24839v1 Announce Type: new Abstract: Agentic data analysis systems produce rich outputs, including code, numerical results, and verbal diagnostics.

By Tian Zheng, Kai-Tai Hsu

llms agents

arXiv AI

Jun 10

The Confident Liar: Diagnosing Multi-Agent Debate with Log-Probabilities and LLM-as-Judge

arXiv:2606. 10296v1 Announce Type: cross Abstract: Multi-agent debate systems are typically evaluated only on whether the final answer is correct, overlooking the quality of the intermediate reasoning that debate is designed to produce.

By Ali Keramati, Justin Cheok, Jacob Horne, Mark Warschauer

llms agents nlp

arXiv AI

Jul 10

Two Axes of LLM Abstention: Answer Correctness and Question Answerability

arXiv:2607. 08456v1 Announce Type: cross Abstract: A model should refuse two different things: answers it would get wrong, and questions it should not answer at all, such as unanswerable ones or ones resting on a false premise.

By Benedikt J. Wagner

llms

Related stories

Reasoning Consistency Scanning: A Framework for Auditing Chain-of-Thought Validity in AI Safety Evaluations

A Reference-Free Score for Detecting Silent Reasoning Failures in Large Language Models

C2-Faith: Benchmarking LLM Judges for Causal and Coverage Faithfulness in Chain-of-Thought Reasoning

Grading the Grader: Lessons from Evaluating an Agentic Data Analysis System

The Confident Liar: Diagnosing Multi-Agent Debate with Log-Probabilities and LLM-as-Judge

Two Axes of LLM Abstention: Answer Correctness and Question Answerability