arXiv Machine Learning
The paper introduces a finite‑dimensional sensitivity framework called the Learning Stability Profile, which captures how perturbations in inputs, parameter initialization, and update mechanisms propagate along a learning trajectory. It presents a Lyapunov criterion that, under regularity, coercivity, and dissipation assumptions, yields uniform or exponentially decaying bounds on linearized transition operators, distinguishing between terminal decay, boundedness, and subexponential growth. The framework is applied to various learning mechanisms—including feedforward networks, residual architectures, stochastic gradient methods, and locally Lipschitz systems—providing spectral, dissipativity, and contraction‑based stability bounds.
The paper investigates how parameter Jacobians influence the stability of network outputs within the framework of network dynamics, learning models, and neural tangent kernels (NTK). It demonstrates that linearized dynamics can be expressed as a semigroup of linear operators on Hilbert spaces, and provides explicit a priori perturbation bounds for fixed‑kernel linearizations in the NTK setting. The authors also offer refinements for task‑specific spaces, ergodic comparison estimates, spectral‑distribution conditions, and extensions to nonautonomous NTK evolutions, supported by worked examples.
By Halyun Jeong, Palle E. T. Jorgensen, Hyun-Kyoung Kwon, Myung-Sin Song, James Tian
arXiv:2611.05869v1 Announce Type: new
Abstract: Temporal-difference (TD) learning underlies many prediction and control algorithms, but can diverge when combined with off-policy learning, bootstrappi...
By Jingyi Jin, Martha White, Hao Wang
arXiv:2608.30431v1 Announce Type: cross
Abstract: By focusing on algorithmic stability as a means of establishing out-of-sample bounds, we provide a system-theoretic interpretation of generalization...
By Filippo Fabiani
arXiv:2606. 04031v1 Announce Type: new Abstract: Coupled gradient descent--where the update of one parameter block depends on another--underlies bilevel optimization, two-time-scale stochastic approximation, and adversarial training.
By Ahanaf Hasan Ariq
arXiv:2506. 01226v3 Announce Type: replace-cross Abstract: We study parameterizations of stabilizing nonlinear policies for learning-based control.
By Nicholas H. Barbara, Ruigang Wang, Alexandre Megretski, Ian R. Manchester
arXiv:2606. 09929v1 Announce Type: cross Abstract: Physical reservoir computing harnesses nonlinear mechanical dynamics but, by convention, freezes the substrate and trains only a linear readout, presuming the substrate is not usefully trainable.
By Caleb Munigety
arXiv:2610.27321v1 Announce Type: cross
Abstract: Gradient descent with momentum can be viewed as a particle moving with friction in the loss landscape. In deep learning the learning rate is usually...
By Alexander Itin
arXiv:2608. 16084v1 Announce Type: new Abstract: Neural autoregressive models have rapidly emerged as powerful emulators of high-dimensional chaotic systems, yet their long-term instability and error growth remain poorly understood, leading to ad-hoc solutions.
By Conrad Ainslie, Pedram Hassanzadeh, Michael W. Mahoney, Ashesh Chattopadhyay
arXiv:2606. 18080v1 Announce Type: new Abstract: Gradient descent in deep learning may operate at the edge of stability (EoS), a regime in which the largest eigenvalue of the loss Hessian hovers near the stability threshold $2/\eta$, where $\eta$ is the learning rate.
By Pierre Marion
arXiv:2605. 08488v2 Announce Type: replace-cross Abstract: We develop a unified Lyapunov-integral quadratic constraint (IQC) framework for establishing uniform stability of first-order accelerated optimization algorithms in the $\beta$-smooth and $\gamma$-strongly convex regime.
By Don Li, Dacian Daescu
The paper derives an exact discrete‑time law that captures how learning‑rate schedules and weight decay interact in scale‑invariant neural networks, showing that a single scalar quantity governs the effective step size. It demonstrates that the balance point between contraction and expansion is intrinsically unstable, leading to recurrent dynamics when using constant learning rates with weight decay. The authors extend this analysis to various optimizers and datasets, confirming the law’s precision and showing that performance peaks sharply at the predicted boundary.
By Hasan Amin, Wei-Kai Chang, Rajiv Khanna
arXiv:2605. 07116v2 Announce Type: replace Abstract: We analyze a neural semi-discrete method for high-dimensional first-order Hamilton-Jacobi-Bellman (HJB) equations with known or learned dynamics.
By Minseok Kim, Yeongjong Kim, Namkyeong Cho, Yeoneung Kim