TinyUDE introduces a solver‑free training framework for Universal Differential Equations (UDEs) that fits a hybrid vector field directly to the first and second time‑derivatives of observed system states using Lie‑Taylor jet matching. The method estimates these derivatives online via Savitzky‑Golay filtering, enabling fully analytic gradients without automatic differentiation. Experiments on damped and chaotic pendulum systems show that TinyUDE matches or surpasses conventional solver‑based baselines while using less than 108 kB of static memory and achieving real‑time on‑device training on an ESP32 microcontroller.
arXiv:2607. 16268v1 Announce Type: new Abstract: Adaptive optimizers such as Adam and AdamW apply the same update rule regardless of whether training is in a chaotic early phase or near convergence.
arXiv:2606. 04736v1 Announce Type: cross Abstract: Physics-informed neural networks (PINNs) have become a promising framework for simulating partial differential equations (PDEs) by embedding physical laws directly into neural network training.
arXiv:2609.24021v1 Announce Type: cross Abstract: Neural operators are data-driven models that learn mappings from inputs that parameterize partial differential equations, such as spatially varying c...
Neural operators are data-driven models that learn mappings from inputs that parameterize partial differential equations, such as spatially varying coefficients, initial conditions, forcing terms, bou...
arXiv:2601. 21542v3 Announce Type: replace-cross Abstract: Flow Matching (FM) models have emerged as a leading paradigm for high-fidelity synthesis.
arXiv:2608. 01997v1 Announce Type: new Abstract: Single-optimizer training is a poor fit for the distinct phases of deep network optimization: adaptive methods handle noisy early gradients well but overshoot flat minima, while SGD with momentum generalizes better in the late phase but converges slowly early on.
arXiv:2605. 29547v2 Announce Type: replace-cross Abstract: Deep learning optimization relies heavily on the assumption of smooth loss landscapes, a condition systematically violated by modern architectures due to non-smooth components such as ReLU activations and quantization operators.
arXiv:2606. 27294v1 Announce Type: cross Abstract: Analog hardware platforms such as coupled oscillators and Analog Ising Machines naturally solve differential equations at a fraction of the energy cost of digital computation, making them attractive for low-power generative modeling, yet a fundamental mismatch exists: modern generative models assume flexible, software-defined dynamics, whereas analog hardware imposes fixed, physics-determined differential equations with limited approximation capacity.
arXiv:2606. 17249v1 Announce Type: cross Abstract: The dominant trajectory of modern machine learning has been to scale up: larger models, larger accelerators, larger memory budgets.
arXiv:2606. 27895v1 Announce Type: cross Abstract: Differentiable partial differential equation (PDE) solvers underpin solver-in-the-loop ML training, gradient-based optimal control, and inverse problems, yet the practical cost of obtaining correct, usable gradients from a given solver on a given problem is largely undocumented.
The paper introduces a framework for evaluating how well learned states from neural PDE solvers can be reused in new contexts. By separating solution accuracy, learning contribution, and numerical utility through paired state comparisons and cost accounting, the authors audit 12 papers and conduct experiments on linear PDE families, Fourier neural operators, and convolutional networks. Their results show that matched state comparisons and downstream computational evidence can lead to significant savings in conjugate‑gradient iterations and demonstrate finite‑budget pretraining value.
GRADSOLVE is an open‑source JAX library that provides fast, exact reverse‑mode gradients for low‑dimensional ordinary differential equation (ODE) ensembles on NVIDIA GPUs. It records the accepted steps of an adaptive solver and differentiates a fixed‑step replay, yielding the exact discrete adjoint at a lower computational cost than traditional checkpointed methods. Benchmarks show that GRADSOLVE’s forward kernel is 2.8× faster than DiffEqGPU.jl, and its gradient computation is 5.6–14.1× faster than Diffrax’s checkpointed adjoint while maintaining matched forward‑state accuracy across multiple GPU generations.