arXiv Machine Learning
Response Renormalization for Critical Deep Equilibrium Models
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Sep 1
Sensitivity-Constrained Neural Operators for Data-Efficient Forward and Inverse Modeling of Partial Differential Equation Systems
The paper introduces Sensitivity‑Constrained Neural Operators (SC‑NOs), which augment standard neural operator training with sampled Jacobian supervision from differentiable solvers or discrete adjoints. By matching selected sensitivities during training, SC‑NOs improve forward prediction accuracy and significantly enhance gradient‑based inverse reconstruction for distributed fields. Experiments on advection–diffusion, RANS–Spalart–Allmaras, high‑dimensional gridded inputs, and a shallow‑water tsunami source‑inversion case demonstrate that SC‑NOs achieve a better accuracy–cost trade‑off and enable near‑real‑time wave‑propagation forecasting from sparse observations.
arXiv AI
Aug 6
The Hamilton-Jacobi Theory of Deep Learning
arXiv:2605. 28983v2 Announce Type: replace-cross Abstract: In this paper, training a neural network is identified, exactly, as a search through Hamilton--Jacobi initial-value problems: each gradient step selects the initial data of a viscous Hamilton--Jacobi equation whose Hopf--Cole propagator best fits the observations; at inference, the input is the spatial point at which that solution is evaluated and the initial condition is already encoded in the weights.
arXiv Machine Learning
5d ago
AYLA: Architecting a loss landscape in shallow neural networks to accelerate feature recovery
AYLA is a loss reparameterization framework that applies a sigmoid‑controlled power‑law transformation to the empirical loss, dynamically adjusting gradient magnitudes without changing stationary points or optimal solutions. By reshaping optimization trajectories, AYLA accelerates descent in flat or saddle‑dominated regions and stabilizes late‑stage training, leading to improved feature recovery in two‑layer tanh networks on synthetic Gaussian data. Experiments show enhanced weight alignment, neuron similarity, activation correlation, and richer internal representations, while mitigating rank collapse and promoting a transition from lazy to active feature‑learning regimes.
arXiv AI
Aug 28
Physics-Informed Stochastic Configuration Machine: A Backpropagation-Free Neural Network with Fast Training for Nonlinear Differential Equations
The paper introduces the Physics-Informed Stochastic Configuration Machine (PI‑SCM), a backpropagation‑free neural network designed for solving nonlinear differential equations. By analytically evaluating local Jacobians, PI‑SCM linearizes the physical loss, enabling optimal weight determination through generalized linear least squares and avoiding iterative nonlinear optimization. The authors present a progressive algorithmic suite—PI‑SC‑I, PI‑SC‑II, and PI‑SC‑III—prove their universal approximation properties, and show through experiments that PI‑SCM achieves high‑fidelity predictions and parameter identification while accelerating training by orders of magnitude compared to standard PINNs.
arXiv Machine Learning
Jul 23
PG-KINN: A Physics-Informed Petrov-Galerkin Kolmogorov-Arnold Network for Solving Forward and Inverse PDEs
arXiv:2607. 20378v1 Announce Type: new Abstract: Physics-informed learning of partial differential equations (PDEs) has been dominated by multilayer perceptrons (MLPs), whose spectral bias and dense parameterization limit both accuracy and interpretability.
arXiv Machine Learning
Jun 4
When Both Layers Learn: Training Dynamics of Representing Linear Models via ReLU Networks
arXiv:2606. 04476v1 Announce Type: new Abstract: In this paper, we study the gradient descent dynamics for jointly training both layers of a one-hidden-layer ReLU network to fit a linear target function.