arXiv AI

CAFE: Causally-Guided Automated Feature Engineering with Multi-Agent Reinforcement Learning

CAFE is a new framework for automated feature engineering that treats the task as a causally‑guided sequential decision process. It first learns a sparse directed acyclic graph to identify causal relationships among features and the target, then uses a multi‑agent deep Q‑learning system to select causal groups and transformation operators while managing feature complexity. On 15 public benchmarks, CAFE outperforms leading AFE baselines, converges faster, and shows markedly lower performance drops under distribution shifts.

arXiv Machine Learning
Aug 19

TabCausal: Pretraining Across Causal Environments for Tabular Causal Discovery

TabCausal is a causal discovery foundation model that learns to map datasets directly to causal graphs by pretraining across diverse causal environments. It uses a dynamic task construction strategy to expose the model to varied graph priors, mechanisms, noise models, dimensions, sample sizes, and intervention regimes, improving transferability from observational and mixed‑interventional data. On large synthetic benchmarks and a new protocol‑guided semantic benchmark, TabCausal outperforms many classical baselines and shows robust structure recovery, especially when interventional evidence is available.

By Zi-Rong Li, Si-Yang Liu, Tian-Zuo Wang, Han-Jia Ye
arXiv Machine Learning
Jul 14

DAG-FM: A Foundation Model for Causal Discovery under Heterogeneous Causal Mechanisms

arXiv:2607. 11510v1 Announce Type: new Abstract: Causal discovery from observational tabular data remains fundamentally challenging, primarily due to the heterogeneity of underlying causal mechanisms and the high-dimensional combinatorial search space of Directed Acyclic Graphs (DAGs).

By Yikang Chen, Zhengkang Guan, Haoyuan Qian, Peng Cui, Yi Yang, Kun Kuang
arXiv Machine Learning
Aug 31

Actionable CBFI: Integrating Structural Decomposition and Causal Counterfactual Recourse for Tabular Machine Learning

The paper introduces Actionable Case-Based Feature Importance (A‑CBFI), a framework that integrates structural causal models with counterfactual recourse for tabular machine learning. A‑CBFI isolates synergistic interaction bottlenecks and releases suppressive structural locks, concentrating over 98.3% of intervention effort on diagnosed root causes. Empirical tests in finance and healthcare show a 76.9% reduction in active human intervention while keeping recourse costs comparable to exhaustive causal methods.

By Sejong Oh
arXiv Machine Learning
Jul 14

FedCausal-Dyn: A Causal-Dynamic Paradigm for Federated Learning under Dynamic Feature Drift

arXiv:2607. 09695v1 Announce Type: new Abstract: This paper addresses the challenging problem of dynamic feature drift in federated learning, where data distributions evolve across clients and over time -- a common scenario in real-world applications like financial technology.

By Kaijie Chen, Alex Johnson, Maria Garcia, Wei Zhang, Daniel Kim
arXiv AI
Oct 5

Mapping and Advancing the Scalability-Accuracy Frontier of Nonlinear Causal Discovery

The paper evaluates four main approaches to scalable nonlinear causal discovery—differentiable, amortized, score‑matching, and combinatorial search—highlighting their distinct accuracy‑runtime trade‑offs. It introduces SPADE, a spline‑based score‑evaluation method that compiles and reuses sufficient statistics, reducing combinatorial search complexity from O(nd³) to O(nd²+d³). Empirical results show SPADE dramatically extends the scalability‑accuracy frontier, solving large‑scale problems with high structural accuracy on both synthetic and real‑world data.

By Hendrik Suhr, Sascha Xu, Jilles Vreeken
arXiv AI
Oct 28

Actionable CBFI: Prescriptive Interventional Diagnosis and Interaction-Aware Escalation for Sparse Causal Algorithmic Recourse

The paper introduces Actionable Case-Based Feature Importance (A-CBFI), a method that transforms the descriptive decomposition of Localized-CBFI into a prescriptive, interventional diagnosis using a structural causal model. A-CBFI selects the cheapest single action when possible and only considers joint actions if a single action cannot achieve the desired outcome. Experiments on ten tabular benchmarks show that A-CBFI matches or exceeds the success of untargeted causal beam search while prescribing fewer direct actions and requiring fewer model evaluations.

By Sejong Oh
arXiv AI
1d ago

Teach the Way It Thinks: Guiding LLM Post-training Data Engineering with Sparse Autoencoders

The paper introduces SAERL, a method that uses Sparse Autoencoders (SAEs) to guide reinforcement learning data engineering for large language models. By mapping training samples into a feature space of interpretable SAE activations, SAERL selects samples with strong, non‑redundant gradient signals, organizes functional regions from easy to hard, and batches from single activation clusters with moderate mixing. Experiments on mathematical reasoning tasks show consistent performance gains across model scales and RL algorithms, with notable improvements on Qwen2.5-Math-1.5B.

By Yi Jing, Zao Dai, Jinwu Hu, Zijun Yao, Lei Hou, Juanzi Li, Xiaozhi Wang
arXiv AI
Sep 10

SAEScientist-Bench: Can AI Agents Conduct Autonomous SAE Interpretability Research?

The paper introduces SAEScientist-Bench, a benchmark that tests whether AI agents can autonomously conduct mechanistic interpretability research using Sparse Autoencoders (SAEs). Agents are tasked with designing contrastive probes and navigating a large feature dictionary in Gemma-2-9B-IT to identify optimal features for a target concept, with performance measured against expert-curated references on activation rank, concept selectivity, and causal steering. Results show that while frontier agents can discover features and outperform controls, they still lag behind expert baselines, especially in causal steering, highlighting both the potential and current limitations of closed-loop autonomous AI research.

By Yuqiao Tan, Shizhu He, Jun Zhao, Kang Liu
arXiv Computation and Language
Sep 22

FLARE: A Full-Lifecycle Dense Supervision Paradigm for Long-Horizon Coding Agents via Generative Reward Model

FLARE introduces a dense supervision paradigm for long‑horizon coding agents, leveraging a Generative Reward Model (GRM) trained via the RADAR diagnostic framework. The GRM provides real‑time, step‑level risk feedback, enabling FLARE to act as an active scaffold that intercepts high‑risk steps during inference and supplies structured signals for post‑training fine‑tuning and reinforcement learning. Experiments show FLARE outperforms existing methods, achieving a 5× reduction in token consumption and significant performance gains in both supervised fine‑tuning and RL settings.

By Jingxuan Xu, Gang Wu, Yanan Wu, Yutao Mou, Songwei Yu, Tianzhuang He, Zhengshuo Gong, Zhao Liu, Zihang Xu, Wenqiang Zhu, Xinping Lei, Weihao Li, Yuhui Bai, Zhongqiu Wang, Yan Wu, Ariel Deng