arXiv AI

Investigating the Interplay between Contextual and Parametric Chain-of-Thought Faithfulness under Optimization

arXiv:2605. 24960v2 Announce Type: replace-cross Abstract: Chain-of-Thought (CoT) faithfulness, i.

arXiv AI

Jul 10

ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation

arXiv:2502. 15543v4 Announce Type: replace-cross Abstract: Large language models (LLMs) integrated with retrieval-augmented generation (RAG) have improved factuality by grounding outputs in external evidence.

By Pengcheng Huang, Zhenghao Liu, Yukun Yan, Haiyan Zhao, Xiaoyuan Yi, Hao Chen, Zhiyuan Liu, Maosong Sun, Tong Xiao, Ge Yu, Chenyan Xiong

llms rag benchmarks

arXiv AI

Aug 3

On the Generalization of Steering Vectors for Chain-of-Thought Faithfulness

arXiv:2607. 29062v1 Announce Type: new Abstract: Model capabilities have improved in large part due to scaling chain of thought.

By Matthew Nguyen, Kyle Cox, Austin Meek, Iv\'an Arcuschin

safety

arXiv AI

3d ago

Making LLMs Say What They Think: Measuring and Improving CoT-Interpretability Alignment

The paper introduces CoT-Interpretability Alignment (CIA), a metric that quantifies how well a large language model’s chain-of-thought (CoT) explanations match its internal reasoning processes. Evaluated on two-hop question answering, hint intervention, and integer multiplication across three LLMs, the study finds limited alignment (44.8–75.9%) and demonstrates that post‑training with a reward combining task accuracy and parametric faithfulness can substantially improve CoT faithfulness without sacrificing accuracy. The authors provide a framework for auditing CoT faithfulness and a pathway to making explicit reasoning more trustworthy, with code and data publicly available.

By Yihuai Hong, Shauli Ravfogel, Chen Zhao, Eunsol Choi

llms nlp safety

arXiv AI

2d ago

Emergent Unfaithfulness: How Alignment Training Causes Language Models to Silently Override Task Faithfulness

arXiv:2610.00568v1 Announce Type: cross Abstract: Large language models are characterized by three key properties: capability, alignment, and faithfulness. Prior work studies the tradeoffs between ca...

By Pardis Sadat Zahraei, Janvijay Singh, Gokhan Tur, Dilek Hakkani-Tur

llms reinforcement-learning safety

arXiv AI

Sep 17

Bypassing the Rationale: Causal Auditing of Implicit Reasoning in Language Models

The paper introduces a causal, layerwise audit method called the CoT Mediation Index (CMI) to evaluate whether chain-of-thought (CoT) prompting truly influences a language model’s internal computation. By comparing performance degradation from patching CoT-token hidden states against matched control patches, the authors find that CoT influence is often confined to narrow reasoning windows and can be nearly absent even when the model produces fluent rationales. The study shows that models explicitly tuned for reasoning exhibit stronger mediation, while Mixture-of-Experts models display more distributed mediation, indicating that CoT faithfulness varies across models and tasks.

By Anish Sathyanarayanan, Aditya Nagarsekar, Aarush Rathore

llms

arXiv Machine Learning

Sep 22

From Concept Alignment to Causal Grounding: An Intervention Test of Chain-of-Thought Faithfulness

arXiv:2609.23065v1 Announce Type: cross Abstract: Chain-of-thought (CoT) can sound plausible yet be unfaithful to the model's underlying reasoning. Most prior work probes CoT faithfulness through inp...

By Qianli Wang, Yilong Wang, Dennis Wei, Jingyi Sun, Simon Ostermann, Pepa Atanasova, Nils Feldhus

llms safety

arXiv AI

Jul 23

Information Discernment in Large Language Models

arXiv:2607. 19355v1 Announce Type: new Abstract: LLMs are increasingly used with external knowledge sources like the internet.

By Joshua Ashkinaze, Laura Kurek, Alina Faisal, Tongyuan Miao, Mariam Joseph, Ceren Budak, Eric Gilbert

llms benchmarks safety

arXiv AI

Jun 2

Isolating LLM Lexical Bias: A Curation-Free Triangulated Metric for Preference-Stage Learning

arXiv:2606. 00334v1 Announce Type: cross Abstract: Various language domains have undergone remarkable changes in recent years; these shifts are largely attributed to the advent of Large Language Models and their misalignment with natural language usage.

By Xiaoyang Ming, Jose Hernandez, Thomas Stephan Juzek

llms reinforcement-learning safety

arXiv AI

Jul 28

What Does Chain-of-Thought Contribute at Probe Time? Evidence for Local Co-Occurrence Activation

arXiv:2605. 26795v2 Announce Type: replace Abstract: Chain-of-thought (CoT) prompting enhances large language model performance, yet what drives these gains remains unclear.

By Xiang Wang, Wei Wei

llms

arXiv AI

Aug 7

Learning When to Trust via Selective Context Preference Optimization

arXiv:2608. 06377v1 Announce Type: cross Abstract: Language models increasingly condition their answers on external signals, and a single misleading one can turn a correct answer wrong.

By Xian Sun, Wei Chow, Yingshuo Wang, Junhao Liu, Wei Gao, Qing Wu, Lingdong Kong

llms reinforcement-learning benchmarks

arXiv AI

Jul 22

Inference-Time Steering for Cross-Lingual Factual Consistency in LLMs

arXiv:2607. 19243v1 Announce Type: cross Abstract: Although Large Language Models (LLMs) demonstrate remarkable multilingual fluency, their internal knowledge representations remain disproportionately biased toward high-resource languages.

By Alexander Manev

llms reinforcement-learning robotics benchmarks safety

arXiv AI

Jun 3

Fully Automated Identification of Lexical Alignment and Preference-Stage Shifts in Large Language Models

arXiv:2606. 03165v1 Announce Type: cross Abstract: The language used by digital chat assistants such as ChatGPT can diverge from human expectations (misalignment).

By Thomas Stephan Juzek, Xiaoyang Ming, Jose A. Hernandez

llms safety

Related stories