arXiv Computation and Language August 25, 2026 By Hoang Pham, Dong Le, Anh Tuan Luu

GRACE: Step-Level Benchmark for Faithful Reasoning over Context

Read the original on arXiv Computation and Language →

GRACE is a step‑level benchmark for evaluating the faithfulness of chain‑of‑thought reasoning over context. It provides human annotations for each step in CoT traces from 10 models across 4 datasets, labeling faithfulness, error category, and natural‑language explanations. The benchmark introduces a data‑driven taxonomy that splits errors into GRACE‑Inference (deductive) and GRACE‑Grounding (factual) tracks, each with four categories, and demonstrates that incorporating step‑level faithfulness signals can improve downstream accuracy and reasoning reliability.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI

Jul 22

Copy Less, Ground More: Overcoming Repetitive Copying in Long-Context Reasoning via Evidence-Aware Reinforcement Learning

arXiv:2607. 19345v1 Announce Type: cross Abstract: Large language models that generate step-by-step reasoning traces have achieved strong performance on complex tasks, and extending them to long-context settings has emerged as an important frontier.

By Lizhe Fang, Weizhou Shen, Tianyi Tang, Yisen Wang

llms reinforcement-learning benchmarks

arXiv AI

Jul 24

Post-Hoc Reasoning in Chain of Thought: Decoding and Steering Pre-Committed Answers

arXiv:2603. 01437v2 Announce Type: replace Abstract: As chain of thought (CoT) has become central to scaling reasoning capabilities in large language models (LLMs), it has also emerged as a promising tool for interpretability, suggesting the opportunity to understand model decisions through verbalized reasoning.

By Kyle Cox, Darius Kianersi, Adri\`a Garriga-Alonso

llms safety

arXiv AI

Sep 21

LogicTrack: Auditing Reasoning Trajectories of Large Language Models with Formal Logic Solvers

arXiv:2609.21492v1 Announce Type: new Abstract: Chain-of-Thought (CoT) reasoning has been shown to improve the performance of large language models (LLMs), yet existing optimization methods largely r...

By Jingyu Hu, Shu Yang, Weiru Liu, Di Wang

llms fine-tuning benchmarks

arXiv AI

Jul 16

Deep Interaction: An Efficient Human-AI Interaction Method for Large Reasoning Models

arXiv:2607. 14049v1 Announce Type: new Abstract: The emergence of Chain-of-Thought (CoT) reasoning has significantly enhanced the ability of large language models (LLMs) to tackle complex, multi-step tasks.

By Hefeng Zhou, Jinxuan Zhang, Jiong Lou, Yuxin Liu, Chaochao Lu, Jingjing Qu, Jie Li

llms

arXiv AI

Jul 14

Valid $\ne$ Necessary: Diagnosing Latent Inefficiency in Chain-of-Thought

arXiv:2607. 11266v1 Announce Type: new Abstract: Chain-of-Thought (CoT) prompting has significantly advanced the reasoning capabilities of Large Language Models (LLMs), yet it often incurs substantial computational costs due to over-reasoning: the generation of redundant, verbose, or irrelevant steps.

By Daeyeop Lee, Hwanjo Yu

llms efficiency benchmarks

arXiv Computation and Language

3d ago

Reasoning Error from Known Fact: Step-Level Self-Consistency Group Relative Policy Optimization for LLMs

The paper introduces Step-level Self-Consistency Group Relative Policy Optimization (SSC‑GRPO), a method that assigns rewards to individual steps in a reasoning trace by evaluating self‑consistency across multiple rollouts. It targets Context‑Sensitive Factual Hallucinations—errors that arise when a model has the correct knowledge but produces false statements due to contextual interference. SSC‑GRPO outperforms previous approaches on mathematical reasoning benchmarks and hallucination leaderboards, offering a new way to detect and reduce hallucinations in large language model reasoning.

By Xiaomeng Hu, Jiaqi Hu, Hao Chen, Qi Zhang, Zhanming Shen, Wentao Ye, Junbo Zhao

llms benchmarks safety

One email a morning, machine-written

Related stories

Copy Less, Ground More: Overcoming Repetitive Copying in Long-Context Reasoning via Evidence-Aware Reinforcement Learning

Post-Hoc Reasoning in Chain of Thought: Decoding and Steering Pre-Committed Answers

LogicTrack: Auditing Reasoning Trajectories of Large Language Models with Formal Logic Solvers

Deep Interaction: An Efficient Human-AI Interaction Method for Large Reasoning Models

Valid $\ne$ Necessary: Diagnosing Latent Inefficiency in Chain-of-Thought

Reasoning Error from Known Fact: Step-Level Self-Consistency Group Relative Policy Optimization for LLMs