arXiv:2607. 23388v1 Announce Type: cross Abstract: As constrained learning becomes increasingly common, models are trained under explicit feasibility requirements to enforce fairness, safety, robustness, regulariza- tion, and physics or logic constraints.
arXiv:2606. 18307v1 Announce Type: cross Abstract: Optimizing the training data distribution for Supervised Fine-Tuning (SFT) dictates the capability of Large Language Models (LLMs).
The paper introduces a control‑variable framework for deep neural networks to mitigate omitted variable bias, particularly shortcut learning where covariates like demographics influence predictions. It refits the final layer of a pre‑trained network using cross‑fitting with ridge penalisation, orthogonalises covariate effects, and marginalises predictions over covariate distributions to achieve unbiased, interpretable results. Experiments on simulated images and neuroimaging data show consistent estimation of true effects and performance close to models trained on unconfounded data.
arXiv:2511. 22823v2 Announce Type: replace-cross Abstract: Weakly supervised learning has emerged as a practical alternative to fully supervised learning when complete and accurate labels are costly or infeasible to acquire.
arXiv:2608. 13133v1 Announce Type: cross Abstract: Distributional shifts arise when the target deployment environment differs from the source environment that generated the training data.
The paper introduces Inverse Knowledge Distillation (IKD), an attack‑agnostic technique that enhances adversarial transferability by maximizing the discrepancy between benign and adversarial prediction distributions on a surrogate model. IKD employs a CE/KL‑equivalent soft‑label objective to push adversarial predictions away from a fixed benign anchor, leveraging Fisher‑sensitive surrogate directions. The authors provide theoretical analysis showing CE and KL induce identical gradients, derive a lower bound on Fisher‑subspace overlap, and demonstrate through extensive ImageNet experiments that IKD consistently improves black‑box attack performance across CNN, ViT, and defended models.
arXiv:2605. 21422v3 Announce Type: replace Abstract: As LLMs continue to scale up, improving training efficiency heavily relies on effective data utilization.
arXiv:2608. 16340v1 Announce Type: cross Abstract: The combination of traditional statistical models and neural network (NN) components into semi-structured hybrid models is an intriguing approach to construct models that, ideally, combine traditional interpretability with the unprecedented flexibility of NNs.
arXiv:2606. 05165v1 Announce Type: new Abstract: Training Data Attribution (TDA) seeks to trace a model's predictions back to its training data.
The paper introduces a method for learning risk scores that remain reliable even when historical data contain unobserved confounders. By treating propensity weights as uncertain and applying sensitivity analysis with Wasserstein distributionally robust optimization, the authors formulate a robust learning problem solvable via an exponential cone program. Experiments on semi‑synthetic UCI data show the approach improves calibration by up to 29.2% over traditional benchmarks and 11.1% over the state of the art, without harming other performance metrics.
arXiv:2605. 27991v2 Announce Type: replace-cross Abstract: Gradient-flow optimization is usually viewed as an algorithmic procedure for minimizing empirical loss, with training duration selected by validation or heuristic early-stopping rules.
arXiv:2509. 09371v2 Announce Type: replace-cross Abstract: Distributionally robust optimization (DRO) protects statistical learning against distributional shifts by optimizing the worst-case performance over a set of perturbed distributions.
arXiv:2606. 05247v1 Announce Type: new Abstract: Enforcing nonlinear inequality constraints in neural networks remains challenging, especially when the output is subject to many coupled constraints.