arXiv AI
Aug 6
Can Post-Training Transform LLMs into Causal Reasoners?
arXiv:2602. 06337v2 Announce Type: replace-cross Abstract: Causal inference is essential for decision-making but remains challenging for non-experts.
Evaluating Causal Reasoning in LLMs: A Case Study in the Challenges of Procedural Generation
The paper investigates whether large language models (LLMs) truly perform formal causal reasoning by testing them on datasets generated by the CLadder framework. It finds that LLMs and even simple models like random forests can achieve high accuracy on CLadder data, largely due to superficial, highly predictive patterns embedded by the generator. The authors argue that strong performance on CLadder likely reflects exploitation of these patterns rather than genuine causal reasoning, and that many commercial LLMs show brittle results when critical text is removed.
Related stories
arXiv AI
Aug 11
TempoBench: Reasoning Execution Without Causal Attribution Is Just Simulation
arXiv:2510. 27544v3 Announce Type: replace Abstract: Current training paradigms, optimized for long-horizon reasoning trace execution, have made Large Language Models (LLMs) excel at pattern matching and forward simulation of reasoning, but underperform at counterfactual causal understanding and reasoning.
arXiv Machine Learning
Sep 10
CausalBN-Bench: A Comprehensive Benchmark for Causal Learning Capability of LLMs
arXiv:2404.06349v3 Announce Type: replace Abstract: The ability to understand causality significantly impacts the competence of large language models (LLMs) in output explanation and counterfactual r...
arXiv AI
Jun 9
TempoBench: Evaluating Temporal Causal Reasoning in Large Language Models
arXiv:2510. 27544v2 Announce Type: replace Abstract: Temporal reasoning involves understanding how systems evolve over time through input-driven state transitions.
Hugging Face Trending Papers
Jul 9
CausalDS: Benchmarking Causal Reasoning in Data-Science Agents
Large language models (LLMs) increasingly act as integrated data-science agents, combining abstract reasoning with advanced tool use. Yet the relevant benchmark landscape largely divides into symbolic causal reasoning benchmarks without realistic data analysis or data analysis benchmarks without a principled causal data-generating structure.
arXiv AI
Jul 10
CausalDS: Benchmarking Causal Reasoning in Data-Science Agents
arXiv:2607. 08093v1 Announce Type: new Abstract: Large language models (LLMs) increasingly act as integrated data-science agents, combining abstract reasoning with advanced tool use.
arXiv AI
Jun 29
CausalFlip: A Benchmark for LLM Causal Judgment Beyond Semantic Matching
arXiv:2602. 20094v2 Announce Type: replace Abstract: As large language models (LLMs) witness increasing deployment in complex, high-stakes decision-making scenarios, it becomes imperative to ground their reasoning in causality rather than spurious correlations.
arXiv AI
Jun 15
C2-Faith: Benchmarking LLM Judges for Causal and Coverage Faithfulness in Chain-of-Thought Reasoning
arXiv:2603. 05167v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used as judges of chain-of-thought (CoT) reasoning, yet it remains unclear whether they can reliably assess process faithfulness rather than merely answer plausibility.
arXiv AI
Jul 7
CausalGame: Benchmarking Causal Thinking of LLM Agents in Games
arXiv:2607. 04293v1 Announce Type: cross Abstract: Building AI Scientist agents with Large Language Models (LLMs) has recently attracted growing attention.
arXiv AI
Oct 27
Discovering and Reasoning of Causality in the Hidden World with Large Language Models
arXiv:2402.03941v4 Announce Type: replace-cross Abstract: Revealing hidden causal variables alongside the underlying causal mechanisms is essential to the development of science. Despite the progress...
arXiv AI
Aug 26
From Causal Plausibility to Causal Reliability: Evaluating LLMs as Calibrated Direct Causal-Edge Classifiers
The study evaluates 12 instruction‑tuned open‑weight LLMs on six causal‑graph benchmarks, testing five prompting strategies and four confidence sources. Findings show that LLMs tend to over‑predict edges, misclassify indirect or reversed edges as direct, and exhibit high over‑confidence, while conventional confidence estimates are unreliable and agreement signals offer limited improvement. The results suggest LLMs should be used as externally validated soft causal priors rather than definitive causal‑structure evidence.
arXiv Machine Learning
Sep 11
CausalArena: Benchmarking Causal Discovery in the Foundation Model Era
CausalArena is a new benchmark designed to evaluate causal discovery methods in the era of foundation models. It unifies synthetic structural causal models (SCMs), semantically grounded SCMs, and formula‑grounded SCMs, while also including real‑world datasets for external validation. Experiments show that performance rankings vary widely across different SCM families and protocols, indicating that strong results on one benchmark do not necessarily transfer to others.