The paper introduces the Directional Influence Function (DIF), a new estimator for measuring how training samples affect the solution of constrained learning problems. DIF incorporates constraints directly by formulating optimality conditions as a variational inequality, thereby avoiding the bias seen in classical influence functions. Experiments on constrained linear regression and fairness‑constrained CNNs show that DIF accurately predicts changes in test loss and aligns closely with actual retraining results.
arXiv:2607. 23388v1 Announce Type: cross Abstract: As constrained learning becomes increasingly common, models are trained under explicit feasibility requirements to enforce fairness, safety, robustness, regulariza- tion, and physics or logic constraints.
The paper introduces a control‑variable framework for deep neural networks to mitigate omitted variable bias, particularly shortcut learning where covariates like demographics influence predictions. It refits the final layer of a pre‑trained network using cross‑fitting with ridge penalisation, orthogonalises covariate effects, and marginalises predictions over covariate distributions to achieve unbiased, interpretable results. Experiments on simulated images and neuroimaging data show consistent estimation of true effects and performance close to models trained on unconfounded data.
arXiv:2606. 18307v1 Announce Type: cross Abstract: Optimizing the training data distribution for Supervised Fine-Tuning (SFT) dictates the capability of Large Language Models (LLMs).
arXiv:2606. 05247v1 Announce Type: new Abstract: Enforcing nonlinear inequality constraints in neural networks remains challenging, especially when the output is subject to many coupled constraints.
The paper argues against using fixed weighted-sum penalties to enforce constraints in deep learning, citing three main issues: non-equivalence of penalized and constrained problems in non-convex settings, difficulty in selecting appropriate penalty coefficients, and the softening of hard requirements. It contends that these problems can lead to non-compliance and relaxed constraints. The authors recommend directly solving the constrained problem when non-negotiable requirements are present.
arXiv:2606. 13803v1 Announce Type: new Abstract: Enforcing functional inequality constraints such as monotonicity and convexity in neural networks is a fundamental challenge in many industrial and scientific applications.
arXiv:2608. 16340v1 Announce Type: cross Abstract: The combination of traditional statistical models and neural network (NN) components into semi-structured hybrid models is an intriguing approach to construct models that, ideally, combine traditional interpretability with the unprecedented flexibility of NNs.
The paper presents a unified framework for regularization-based robust reinforcement learning by deriving upper bounds on the performance gap between nominal and worst-case policies. These bounds are expressed as a regularization objective plus a KL-divergence penalty, explaining why KL penalties enhance robustness. The authors reformulate robust training as a constrained optimization problem, updating the Lagrange multiplier jointly with the policy to automatically tune regularization, and validate the approach with extensive adversarial evaluations on continuous control tasks.
arXiv:2609.34426v2 Announce Type: replace Abstract: This paper addresses the problem of safe offline reinforcement learning, which involves training a policy to satisfy safety constraints using an of...
arXiv:2605. 30456v2 Announce Type: replace Abstract: Many learning tasks in science and engineering are characterized by sparse datasets, which limits the effectiveness of purely data-driven approaches.
The paper introduces a new framework for activation steering in large language models, distinguishing between the steering direction (estimand) and the method used to recover it (estimator). It shows that most existing techniques can be viewed as plug‑in estimators of the average gradient of an outcome regression, and proposes a complementary estimator based on the Stein score to directly target the gradient. By combining these approaches, the authors achieve an orthogonalized estimator with double‑robustness properties and demonstrate its effectiveness on benchmark datasets.
FCx is a new algorithm that generates counterfactual explanations while explicitly enforcing feasibility constraints. It uses a modified Variational Autoencoder with a multi‑factor loss to produce realistic, low‑cost counterfactuals that satisfy both hard constraints supplied by users and soft constraints inferred via causal inference. Experiments on four public datasets demonstrate that FCx matches state‑of‑the‑art performance across multiple metrics while guaranteeing feasibility.