CausalArena is a unified, evolvable benchmark designed to evaluate causal discovery methods across diverse structural causal models (SCMs). It incorporates synthetic SCMs for controlled structural variation, semantic operational SCMs for human-auditable environments, and formula-grounded SCMs to test discovery under explicit scientific mechanisms, along with real-world datasets for external validity. Experiments show that performance rankings vary significantly across SCM families and protocols, indicating that strong results on one benchmark do not generalize to others, especially in the context of causal discovery foundation models.
CausalArena is a new benchmark designed to evaluate causal discovery methods in the era of foundation models. It unifies synthetic structural causal models (SCMs), semantically grounded SCMs, and formula‑grounded SCMs, while also including real‑world datasets for external validation. Experiments show that performance rankings vary widely across different SCM families and protocols, indicating that strong results on one benchmark do not necessarily transfer to others.
By Zi-Rong Li, Si-Yang Liu, Tian-Zuo Wang, Han-Jia Ye
arXiv:2603. 05167v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used as judges of chain-of-thought (CoT) reasoning, yet it remains unclear whether they can reliably assess process faithfulness rather than merely answer plausibility.
By Avni Mittal, Rauno Arike
The study evaluates 12 instruction‑tuned open‑weight LLMs on six causal‑graph benchmarks, testing five prompting strategies and four confidence sources. Findings show that LLMs tend to over‑predict edges, misclassify indirect or reversed edges as direct, and exhibit high over‑confidence, while conventional confidence estimates are unreliable and agreement signals offer limited improvement. The results suggest LLMs should be used as externally validated soft causal priors rather than definitive causal‑structure evidence.
By Amit Kumar, Elnur Adl Zarabi, Suranjana Trivedy, Zhiqian Chen, Lei Zhang, Kaiqun Fu, Taoran Ji
arXiv:2601. 13735v2 Announce Type: replace Abstract: Probabilistic confidence metrics are increasingly adopted as proxies for reasoning quality in Best-of-N selection, under the assumption that higher confidence reflects higher reasoning fidelity.
By Hojin Kim, Jaehyung Kim
arXiv:2404.06349v3 Announce Type: replace
Abstract: The ability to understand causality significantly impacts the competence of large language models (LLMs) in output explanation and counterfactual r...
By Yu Zhou, Xingyu Wu, Jibin Wu, Liang Feng, Kay Chen Tan
The paper introduces a causal, layerwise audit method called the CoT Mediation Index (CMI) to evaluate whether chain-of-thought (CoT) prompting truly influences a language model’s internal computation. By comparing performance degradation from patching CoT-token hidden states against matched control patches, the authors find that CoT influence is often confined to narrow reasoning windows and can be nearly absent even when the model produces fluent rationales. The study shows that models explicitly tuned for reasoning exhibit stronger mediation, while Mixture-of-Experts models display more distributed mediation, indicating that CoT faithfulness varies across models and tasks.
By Anish Sathyanarayanan, Aditya Nagarsekar, Aarush Rathore
arXiv:2609.11170v2 Announce Type: replace
Abstract: Can a trace explaining model execution also compute the changes needed for a specified alternative prediction? We propose trace-guided intervention...
By Minwoo Yu, Young-guk Ha
arXiv:2508.15754v2 Announce Type: replace-cross
Abstract: Tool use is often assumed to monotonically improve reasoning, where external evidence is expected to help when relevant and be ignored when i...
By Yufeng Zhao, Junnan Liu, Hongwei Liu, Dongsheng Zhu, Yuan Shen, Songyang Zhang, Kai Chen
arXiv:2607. 15281v1 Announce Type: new Abstract: Causal and intervention-based question answering is fundamental to advancing large language models (LLMs) toward reasoning beyond surface-level correlations and understanding underlying causal mechanisms.
By Su Lan, Xuefei Yin, Yanming Zhu, Alan Wee-Chung Liew
CIDER-FM is a causal foundation model that combines finite observational data with surrogate-interventional datasets to predict target conditional interventional distributions more accurately than using observational data alone. It employs an intervention-aware representation and hierarchical three‑axis attention to integrate information across variables, samples, and experimental regimes. Experiments on synthetic graphs, simulated data, and real‑world Causal Chambers data show that incorporating experimental context improves CID prediction performance.
By Yuche Gao, Arik Reuter, Siyuan Guo, Anish Dhir, Bernhard Sch\"olkopf, Adrian Weller
arXiv:2607. 11816v1 Announce Type: new Abstract: Causal discovery algorithms learn a network that describes the causal dependencies among random variables.
By Bijan Mazaheri, Jiaqi Zhang, Caroline Uhler