TASTE (Task-Aware STEin operators) is a framework that links distribution shift to a model’s input sensitivity using Stein operators. It provides a geometric interpretation of shift as a projection onto the model’s sensitivity field, offers coordinate‑wise localisation, and delivers interpretable per‑pixel diagnostics for image data. Experiments on Gaussian shifts, MNIST, and CIFAR‑10/ImageNet‑1K benchmarks show that TASTE closely tracks task degradation and performs competitively with existing baselines, with gains varying by shift regime.
By Micha{\l} Kozyra, Gesine Reinert
The paper introduces StablePDENet, a physics-informed adversarial training framework that regularizes the residual sensitivity of neural operators to improve stability against input perturbations. The method formulates operator learning as a min–max optimization, where a physics-based projected‑gradient adversary generates perturbations and the outer objective combines the attacked physics loss with a normalized residual‑sensitivity penalty. Experiments on benchmark problems show that StablePDENet outperforms PI‑DeepONet and its adversarial variant in accuracy under adversarial attacks while maintaining competitive performance on clean data, and it also enhances generalization and highlights the distinction between model sensitivity and intrinsic operator ill‑conditioning.
By Chutian Huang, Chang Ma, Kaibo Wang, Yang Xiang
arXiv:2605. 29547v2 Announce Type: replace-cross Abstract: Deep learning optimization relies heavily on the assumption of smooth loss landscapes, a condition systematically violated by modern architectures due to non-smooth components such as ReLU activations and quantization operators.
By Ruoran Xu, Borong She, Xiaobo Jin, Qiufeng Wang
arXiv:2505. 13196v3 Announce Type: replace-cross Abstract: We introduce Velocity-Regularized Adam (VRAdam), a physics-inspired optimizer for training deep neural networks that draws on ideas from quartic terms for kinetic energy with its stabilizing effects on various system dynamics.
By Pranav Vaidhyanathan, Lucas Schorling, Natalia Ares, Maike Osborne
arXiv:2608. 10374v1 Announce Type: new Abstract: Training neural networks to jointly predict mean and uncertainty estimates from noisy observations can be unstable, prompting a series of independent stabilization efforts.
By Sumedh Vemuganti, Nickvash Kani
arXiv:2608. 07924v1 Announce Type: cross Abstract: Drifting models are a recent class of one-step generative models that evolve the model distribution during training using a predefined sample-based drift field.
By Drake Brown, Yuhao Huang, Shih-Hsin Wang, Bao Wang
Training neural networks to jointly predict mean and uncertainty estimates from noisy observations can be unstable, prompting a series of independent stabilization efforts. We argue that these interventions highlight a common underlying issue where gradient steps are poorly aligned with the geometry of the loss landscape.
arXiv:2606. 07835v1 Announce Type: new Abstract: A fundamental tension exists in the large-step inference of diffusion models via their deterministic probability flow ordinary differential equation (PF-ODE) trajectories, which we identify as the contractivity trap: efficient inference favors large step sizes, while aggressive steps and highly expressive denoisers can undermine contraction-based stability certificates for error suppression.
By Shigui Li, Delu Zeng
The paper introduces a unified matrix‑spectral framework that treats deep neural networks as data‑dependent products of linear operators, enabling the analysis of stability and interpretability through spectral quantities. It defines a Global Matrix Stability Index that aggregates spectral information from Jacobians, parameter gradients, Neural Tangent Kernel operators, and loss Hessians into a single stability scale, and shows that spectral entropy provides a more realistic measure of sensitivity than worst‑case bounds. Experiments on MNIST, CIFAR‑10, and CIFAR‑100 demonstrate that modest spectral regularization can significantly improve attribution stability without large changes to global spectral summaries.
By Ronald Katende
arXiv:2505. 11766v4 Announce Type: replace Abstract: Neural Operators (NOs) are powerful architectures for learning mappings between function spaces.
By Haoze Song, Zhihao Li, Xiaobo Zhang, Zecheng Gan, Zhilu Lai, Wei Wang
arXiv:2608. 18237v1 Announce Type: cross Abstract: Estimating the difference of two Stein's score functions is a fundamental problem in generative modeling.
By Chenghan Xie, Jose Blanchet, Renyuan Xu
arXiv:2607. 23940v1 Announce Type: new Abstract: Physics-Informed Neural Networks (PINNs) solve differential equations by minimizing the residual of a nonlinear operator over a neural parameterization of the solution.
By Pavlos Protopapas, Kaylee Vo