arXiv Computation and Language
3d ago

From Passive Response to Proactive Correction: Enhancing LLM Robustness Against Input Fact Perturbations

The paper introduces DEDUCE, a three‑stage framework that turns large language models into proactive error correctors by detecting input fact errors, devising correction strategies, and delivering reliable answers. It also presents MisFactQA, a dataset of factual errors, and new metrics for robustness evaluation. Experiments on TruthfulQA, FalseQA, and MisFactQA show significant gains in accuracy and error correction across Qwen, LLaMA, and Gemma models.

By Ping Wang, Xiangguo Sun, Bingbing Xu, Guocong Li, Xiaofeng Meng
arXiv Computation and Language
5d ago

GRACE: Step-Level Benchmark for Faithful Reasoning over Context

GRACE is a step‑level benchmark for evaluating the faithfulness of chain‑of‑thought reasoning over context. It provides human annotations for each step in CoT traces from 10 models across 4 datasets, labeling faithfulness, error category, and natural‑language explanations. The benchmark introduces a data‑driven taxonomy that splits errors into GRACE‑Inference (deductive) and GRACE‑Grounding (factual) tracks, each with four categories, and demonstrates that incorporating step‑level faithfulness signals can improve downstream accuracy and reasoning reliability.

By Hoang Pham, Dong Le, Anh Tuan Luu