REALHOP introduces a behavioral auditing framework to assess multi‑hop reasoning by measuring the Behavioral Necessity Rate (BNR), which quantifies how often removing targeted evidence prevents correct answers. Across five benchmarks, the framework reveals a wide gap between annotated reasoning chains and actual evidence dependence, with panel‑mean BNR ranging from 16.6% to 48.9%. By re‑binding entities, factorizing relations, adding competing paths, and placing evidence at traceable locations, REALHOP raises BNR dramatically—from 27.4% to 94.4% on MuSiQue questions—while maintaining high overall accuracy and improving performance on long‑context tasks.
By Jiawen Tao, Xiaokun Yuan, Yaoming Li, Chenxu Liu, Mengzhou Wu, Tong Yang, Maxm Pan
arXiv:2608. 02011v2 Announce Type: replace Abstract: Agentic retrieval-augmented generation (RAG) systems can fail before evidence-conditioned reasoning is tested: an agent may retrieve candidate snippets but finalize without inspecting them.
By Daeyoung Roh, Donghee Han
arXiv:2608. 14375v1 Announce Type: new Abstract: Multi-agent reasoning systems often use agreement, confidence, or automated scores to decide which messages should shape a final answer.
By Chih-Hsuan Yang, Anjir Ahmed Chowdhury, Cheng-Hau Yang, Weijian Zheng, Fernando Llorente, Xiaolong Ma, Xinyang Li, Eliu A. Huerta, Ian T. Foster, Rajeev Thakur
The paper investigates whether giving AI monitors access to the final answer improves their ability to verify reasoning. Using 237 step‑by‑step solutions to physics exam questions, the authors found that answer access mainly helps monitors detect inconsistencies with the final answer rather than independently checking the reasoning. Certification of the answer increased overall accuracy and error localization but reduced the ability to flag critical traces where the answer was correct but the reasoning was flawed.
By Will Yeadon, Sergio Ju\'arez, Paul Mackay, T. J. Dowling, Elise Agra, Oto-obong Inyang, Arin Mizouri, Craig P. Testrow
TraceForge introduces a two‑stage training method that generates reasoning traces in the style of a target model family, using supervised learning followed by adversarial refinement. The system produces traces that are judged as authentic with scores comparable to real traces, and when applied to teacher answers, it improves performance on coding benchmarks such as HumanEval+ and MBPP+. The approach addresses the limitation of existing models that only return answers without detailed reasoning, enabling more transparent and learnable outputs.
By Yan Wang
The paper investigates whether latent chain-of-thought (latent CoT) models truly perform reasoning by examining twenty-one model-task configurations across different training recipes, tasks, and backbones. Using chain deletion, content replacement, probes, and trajectory analysis, the authors find that on logical tasks the latent chain often contains an answer before the chain starts and can be deleted with little effect, while on arithmetic tasks the chain’s content is essential. The study introduces a taxonomy that distinguishes what a chain represents from how a model uses it, clarifying that readable latent states alone do not prove reasoning.
By Jingjie Qiu, Sheldon Yu, Xunyi Jiang, Jiajin Liu, Zihan Huang, Zhendong Chu, Tong Yu, Julian McAuley, Junda Wu
arXiv:2607. 04572v1 Announce Type: new Abstract: Large language model (LLM) tutors often produce fluent step-by-step explanations, but a correct and pedagogically formatted response does not guarantee that the answer was derived from the student-facing problem.
By Bonan Shen, Dingyan Shang, Youting Wang, Tao Ning
arXiv:2606. 30128v1 Announce Type: new Abstract: Chain-of-thought (CoT) prompting improves LLM reasoning, but the source is contested: do the intermediate steps help because they carry useful semantic content, or because conditioning on more tokens buys extra computation before the model commits to an answer?
By Wenlong Wang, Fergal Reid
arXiv:2608. 03401v1 Announce Type: cross Abstract: Large language models often reason at length before answering, increasing cost and latency.
By Francesca Carlon, Vincent Ginis, Andres Algaba
arXiv:2608. 12877v1 Announce Type: new Abstract: Multi-hop fact verification, which verifies claims by reasoning over multiple pieces of evidence, is critical for combating misinformation on social media yet remains highly challenging.
By Runze Zhao, Zixin Tang, Xiaoshuai Hao, Leyuan Chang, Xiaopeng Fu, Boyu Qiao, Dongyang Zhang
VeriPhy is an auditable physical‑verification system that evaluates generated video by compiling prompts into typed physical obligations and a statically validated execution plan before any frames are observed. During execution, it gates calls to frozen low‑level experts (e.g., segmentation, tracking, counting, depth, OCR, audio‑event detection) and returns provenance‑carrying evidence records, which are mapped to a three‑valued state (supported, contradicted, unknown) with full traceability. On a 1,500‑clip corpus of human‑annotated flaw records, VeriPhy accounts for 228 failures out of 304, outperforming a published question‑decomposition evaluator that accounts for 164, while also providing auditable evidence for each verdict.
arXiv:2610.13996v1 Announce Type: cross
Abstract: Reinforcement learning with verifiable rewards improves mathematical reasoning when a final answer can be checked, but a correct answer does not cert...
By Hamed Mahdavi (Pennsylvania State University), Pouria Mahdavinia (Pennsylvania State University), Majid Farhadi (Google), Vasant Honavar (Pennsylvania State University)