The paper argues against using fixed weighted-sum penalties to enforce constraints in deep learning, citing three main issues: non-equivalence of penalized and constrained problems in non-convex settings, difficulty in selecting appropriate penalty coefficients, and the softening of hard requirements. It contends that these problems can lead to non-compliance and relaxed constraints. The authors recommend directly solving the constrained problem when non-negotiable requirements are present.
arXiv:2606. 13803v1 Announce Type: new Abstract: Enforcing functional inequality constraints such as monotonicity and convexity in neural networks is a fundamental challenge in many industrial and scientific applications.
arXiv:2605.24437v2 Announce Type: replace Abstract: We present a novel framework for embedding hard constraint satisfaction into neural network (NN) architectures, specifically feedforward neural net...
arXiv:2608.24743v1 Announce Type: new Abstract: Existing linear program (LP) and semidefinite program (SDP) relaxations for rectified linear unit (ReLU) neural network (NN) verification yield overly-...
arXiv:2606. 29521v1 Announce Type: new Abstract: Deep learning problems rarely involve objectives that are equal in importance.
HUANet is a deep neural network architecture that unrolls the Alternating Direction Method of Multipliers (ADMM) into a trainable model for accelerating parametric constrained convex optimization. It embeds a hard‑constrained neural network in each ADMM iteration, using a differentiable correction stage to enforce affine equalities of the primal subproblem. The method also incorporates first‑order optimality conditions into a self‑supervised training loss, and numerical experiments on benchmark problems and a control application demonstrate its effectiveness in speeding up constrained convex optimization.
The paper introduces the Directional Influence Function (DIF), a new estimator for measuring how training samples affect the solution of constrained learning problems. DIF incorporates constraints directly by formulating optimality conditions as a variational inequality, thereby avoiding the bias seen in classical influence functions. Experiments on constrained linear regression and fairness‑constrained CNNs show that DIF accurately predicts changes in test loss and aligns closely with actual retraining results.
Existing linear program (LP) and semidefinite program (SDP) relaxations for rectified linear unit (ReLU) neural network (NN) verification yield overly-conservative safety guarantees due to significant...
CG4AI is a column generation framework that trains AI models while enforcing linear constraints on their outputs. It constructs a convex combination of models, using a master linear program to set mixture weights and a pricing subproblem to generate new models guided by dual variables, focusing on the most violated constraints. The method is applied to MNIST digit classification—demonstrating constraint learning, adversarial robustness, error correction, and output relabeling—and to multi‑commodity flow routing, achieving feasible predictors with higher accuracy than single‑model baselines.
The paper presents a unified framework for regularization-based robust reinforcement learning by deriving upper bounds on the performance gap between nominal and worst-case policies. These bounds are expressed as a regularization objective plus a KL-divergence penalty, explaining why KL penalties enhance robustness. The authors reformulate robust training as a constrained optimization problem, updating the Lagrange multiplier jointly with the policy to automatically tune regularization, and validate the approach with extensive adversarial evaluations on continuous control tasks.
arXiv:2605.18618v3 Announce Type: replace-cross Abstract: Constrained Machine Learning (CML) enables fairness-aware training, physics-informed neural networks, and integration of symbolic domain know...
arXiv:2608.09523v2 Announce Type: replace Abstract: Stochastic gradient descent (SGD) and its variants train deep neural networks (DNNs) effectively in practice, yet classical optimization theory doe...
arXiv:2608. 09523v1 Announce Type: new Abstract: Deep neural network (DNN) training with stochastic gradient descent (SGD) and its variants achieves strong empirical performance, yet classical optimization theory does not fully explain this success.