The paper introduces a control‑variable framework for deep neural networks to mitigate omitted variable bias, particularly shortcut learning where covariates like demographics influence predictions. It refits the final layer of a pre‑trained network using cross‑fitting with ridge penalisation, orthogonalises covariate effects, and marginalises predictions over covariate distributions to achieve unbiased, interpretable results. Experiments on simulated images and neuroimaging data show consistent estimation of true effects and performance close to models trained on unconfounded data.
arXiv:2606. 21806v2 Announce Type: replace Abstract: Deep generative models reproduce the observational distribution of their training data, inheriting any spurious associations it contains.
arXiv:2607. 23388v1 Announce Type: cross Abstract: As constrained learning becomes increasingly common, models are trained under explicit feasibility requirements to enforce fairness, safety, robustness, regulariza- tion, and physics or logic constraints.
arXiv:2609.14124v1 Announce Type: cross Abstract: Medical image analysis is often hindered by biased datasets, which can lead to biased models and limited clinical applicability. A promising strategy...
The paper introduces the Directional Influence Function (DIF), a new estimator for measuring how training samples affect the solution of constrained learning problems. DIF incorporates constraints directly by formulating optimality conditions as a variational inequality, thereby avoiding the bias seen in classical influence functions. Experiments on constrained linear regression and fairness‑constrained CNNs show that DIF accurately predicts changes in test loss and aligns closely with actual retraining results.
arXiv:2604. 11704v2 Announce Type: replace-cross Abstract: Deep Neural Networks are highly susceptible to shortcut learning, frequently memorizing low-dimensional spurious correlations instead of underlying causal mechanisms.
The paper introduces CauVaDE, a causal generative model that represents an unobserved confounder as a discrete latent cluster, enabling accurate approximation of both observational and interventional image distributions. It demonstrates that this representation can capture diverse causal explanations without sacrificing fit to the data. A novel causal bootstrap method is proposed to vary confounder entropy, producing a family of distinct interventional image distributions.
The paper systematically evaluates regression objectives for tabular deep learning across many datasets and neural backbones, finding that most alternatives to Mean Squared Error (MSE) offer limited or inconsistent gains, while quantile loss consistently improves performance. It identifies a limitation of global early stopping: different noise levels in target regions reach optimal performance at different epochs, causing a compromise. To address this, the authors propose a heteroskedasticity‑aware extension of quantile loss using input‑dependent label smoothing, which outperforms standard quantile loss on several backbones.
arXiv:2602.01718v2 Announce Type: replace Abstract: Predicting generalization from quantities available before target-test evaluation remains a central challenge in deep learning. The systematic benc...
The paper investigates when supervised learning can recover the true latent variables that generate data, focusing on two causal settings: generative (labels cause latents) and discriminative (latents cause labels). It shows that if the representation space has at least as many dimensions as the latent space, optimal encoders are linear; otherwise, identifiability fails and encoders become nonlinear. Additional constraints—such as non‑negative activations in the generative case or sparsity and spark conditions in the discriminative case—enable recovery of individual latent variables, linking nonlinear ICA, compressed sensing, and neural network interpretability.
arXiv:2608. 01352v1 Announce Type: new Abstract: Estimating causal effects from real-world spatiotemporal data is challenging due to hidden confounders and interference.
The paper introduces a method for learning risk scores that remain reliable even when historical data contain unobserved confounders. By treating propensity weights as uncertain and applying sensitivity analysis with Wasserstein distributionally robust optimization, the authors formulate a robust learning problem solvable via an exponential cone program. Experiments on semi‑synthetic UCI data show the approach improves calibration by up to 29.2% over traditional benchmarks and 11.1% over the state of the art, without harming other performance metrics.
The paper investigates how learned multimodal representations of high‑dimensional health data can fail to preserve causal information, even when trained to reconstruct the data. It establishes conditions under which a representation is sufficient for causal identification, shows that interventional mediation requires stricter preservation than counterfactual comparison, and proves that reconstruction‑based objectives cannot certify sufficiency. The authors decompose the error of cross‑fitted doubly robust estimators into a first‑order representation bias and a second‑order nuisance‑estimation remainder, demonstrating that coverage can deteriorate with larger samples if the representation is insufficient. They propose a practical workflow that combines modality‑specific variational autoencoders, cross‑fitted doubly robust estimation, locked phenotypes, selective pathway analysis, portable representations, and diagnostics for representation insufficiency, exposure leakage, and sensitivity, and validate it in simulations where a rare confounder’s omission leads to severely under‑covered indirect‑effect intervals.