arXiv:2607. 26787v1 Announce Type: new Abstract: Markov Decision Processes (MDPs) are widely used as decision-making models, commonly specified over factored state spaces through state variables and their valuations.
Probabilistic model checking for Markov decision processes (MDPs) provides quantitative guarantees, but often offers limited insight into why undesired outcomes occur. Probability-raising (PR) causality addresses this by identifying states whose visitation increases the probability of reaching designated states.
arXiv:2606. 29681v1 Announce Type: new Abstract: Probabilistic model checking for Markov decision processes (MDPs) provides quantitative guarantees, but often offers limited insight into why undesired outcomes occur.
arXiv:2511. 19849v2 Announce Type: replace-cross Abstract: Recurrence objectives, where a target region must be visited infinitely often, are a fundamental class of specifications for Markov decision processes (MDPs) and form the core of $\omega$-regular and linear temporal logic (LTL) objectives.
The paper introduces Temporal-Logic-based Causal Diagrams (TL-CDs) for reinforcement learning tasks that involve temporally extended goals. TL-CDs encode causal relationships among environmental properties, complementing deterministic finite automata that model rewards. By leveraging TL-CDs, the authors design an RL algorithm that can predict expected rewards early, leading to significantly reduced exploration and faster convergence to optimal policies.
arXiv:2609.05459v1 Announce Type: new Abstract: Reinforcement learning and large language models often struggle to accurately capture the causal mechanics of game environments. Standard reinforcement...
arXiv:2510. 27544v3 Announce Type: replace Abstract: Current training paradigms, optimized for long-horizon reasoning trace execution, have made Large Language Models (LLMs) excel at pattern matching and forward simulation of reasoning, but underperform at counterfactual causal understanding and reasoning.
arXiv:2608. 19831v1 Announce Type: new Abstract: Causal Bayesian networks (CBNs) and structural causal models (SCMs) are the dominant frameworks for graphical causal reasoning, but they cannot adequately represent all real-world causal systems.
The paper introduces a formal framework that uses symmetries in data to keep causal mechanisms invariant, providing a simple and general mathematical language for causal reasoning. It outlines how to describe models and queries, and presents strategies for rigorously identifying causal effects from data within this framework. The approach reproduces known results for IID data and extends causal analysis to non‑IID settings, complex queries beyond do‑ or soft‑interventions, and incorporates missing data, transfer, and robustness considerations.
arXiv:2603. 06946v2 Announce Type: replace Abstract: Many distributional quantities in reinforcement learning are intrinsically joint across actions, including distributions of gaps and probabilities of superiority.
arXiv:2608.24986v2 Announce Type: replace Abstract: Robust POMDPs (RPOMDPs) generalize classical POMDPs to the setting where exact transition probabilities are not known -- rather, they are only know...
arXiv:2607. 16717v1 Announce Type: cross Abstract: This paper proposes a causal independence principle for value -- the value Causal Markov Condition (v-CMC) -- and develops the conceptual and mathematical foundations of a "causal value theory" linking causality and utility.
arXiv:2608. 15645v1 Announce Type: new Abstract: Transporting a causal conclusion from a source study population to a target one is a fundamental problem in causal inference.