The paper investigates whether the tokens highlighted by large language models (LLMs) as key to their predictions are behaviorally consistent when those tokens are manipulated. Using sufficiency, comprehensiveness, and counterfactuality metrics, the authors evaluate closed‑source GPT‑4o‑mini and open‑source models Gemma3, Granite8B, and DeepSeek on IMDB and Steam review datasets. Results show that GPT‑4o‑mini consistently follows expected patterns, Gemma3 and Granite8B maintain sufficiency but falter under stricter interventions, and DeepSeek variants exhibit structural inconsistencies, indicating that explanation reliability varies across model families and sizes.
By Alisson Rosa Pereira, Bruno Iochins Grisci
The paper introduces a method for analyzing how interventions in a prompt affect a language model’s reasoning by using behavioral counterfactuals. It trains a module to edit the residual stream so that the next-token distributions for a fixed response match between a source and a counterfactual prompt, outperforming standard activation reconstruction. The resulting attribution maps and visualizations help identify causally important locations for counterfactual recovery and guide hypothesis testing with activation patching, as demonstrated in a case study on Gemma-4-E4B-it and DeepSeek-R1-0528-Qwen3-8B.
By Shawn Im, James Oldfield, Adel Bibi, Sharon Li
Faithfully understanding the internal process that occurs as models form their chain of thought is an important problem with the rise in model capabilities. However, this remains challenging due to th...
arXiv:2603. 28371v2 Announce Type: replace-cross Abstract: When an agent can articulate why something works, we typically take this as evidence of genuine understanding.
By Camilo Chac\'on Sartori
arXiv:2609.08216v1 Announce Type: new
Abstract: Agentic AI systems built on large language models fail in two persistent ways that scaling does not fix: they break under distribution shift, and they...
By Arun Vignesh Malarkkan, Xinyuan Wang, Yanjie Fu
The paper introduces a refined evaluation framework for chain‑of‑thought (CoT) monitoring in large language models, showing that the commonly used minimal‑criterion sensitivity metric overestimates causal detection of targeted behaviors. It establishes upper and lower bounds on causal detection rates and uses these to assess monitorability. The authors then create a dataset of control–intervention task pairs and apply three training methods—off‑policy self‑distillation, reinforcement learning, and on‑policy self‑distillation—to produce model variants that consistently improve CoT monitorability across various configurations.
By Qiang Fu, Xinyue Li, Andre Wibisono, Arman Cohan
AdaGuard is an adaptive guard model that evaluates language‑model agent trajectories against user‑defined policies. The authors created the AdaptiveSafety dataset, containing 10,939 training and 1,000 test examples with policy rules, counterfactuals, explanations, and violated rules. Using supervised pre‑training followed by the SafePO reinforcement‑learning algorithm, they built 0.6B, 4B, and 8B guard models, with the 4B version achieving 89.30% accuracy on AdaptiveSafety and 71.82% on DynaBench.
By Yunhao Feng, Yifan Ding, Yuxiang Xie, Zheng Li, Mingrui Lao, Zeyuan Wang, Yanming Guo
arXiv:2606. 28615v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed in high-stakes domains, where free-text explanations such as chain-of-thought and post-hoc rationales are used to justify model outputs.
By Nhi Nguyen, Shauli Ravfogel, Rajesh Ranganath
arXiv:2608.30956v1 Announce Type: cross
Abstract: Counterfactual explanations (CEs) are widely used in explainable artificial intelligence (AI) to show how a model's outputs would change if the input...
By Mattia Cerrato, Otto Sahlgren, Xenia Heilmann
arXiv:2610.09237v1 Announce Type: cross
Abstract: Scientific studies of language agents need behavioral variables that support hypotheses across tasks and models. We formulate this research problem a...
By Tianqiang Yan
MEA is a multi‑agent framework that automates the selection and configuration of explanation tools for machine learning models across tabular, text, and vision data. A Proposer agent chooses appropriate tools based on the question and modality, while an Actor agent is trained end‑to‑end for faithfulness, producing natural‑language explanations. Experiments show that MEA outperforms existing post‑hoc explainers and closed‑source baselines, achieving significant faithfulness gains on six datasets.
By Yuyang Cheng, Raghav Kaushik Ravi, Srivarshinee Sridhar, Sriparna Saha, Akash Ghosh, Chirag Agarwal
Scientific studies of language agents need behavioral variables that support hypotheses across tasks and models. We formulate this research problem as learning and testing a hierarchy of trajectory ab...