The paper introduces the Retain-and-Repair Neural Operator (R$^2$NO), a method that adapts neural operators pretrained on numerical simulations to real-world data while preserving useful pretrained structure. R$^2$NO first fine‑tunes the pretrained operator on real data, freezes it to serve as a source prediction, and then employs a shared repair module that learns refinements via orthogonal Fourier projections and a spectral ensemble of ridge regressions. The approach allows refinement depth to vary across frequency magnitude, orientation, and channels, and consistently outperforms full fine‑tuning and iterative refinement on RealPDEBench systems across six backbones.
By Woojin Cho, Junghwan Park
arXiv:2606. 16602v1 Announce Type: new Abstract: Neural operator models trained on simulation data often lose accuracy when applied to experimental measurements due to the sim-to-real gap.
By Changjian Zhou, Junfeng Fang, Negin Yousefpour, Peng Wu, Bin Yan, Guillermo A Narsilio
The paper investigates why latent neural surrogate solvers, which compress physical system dynamics into a lower‑dimensional space, often fail during long‑horizon autoregressive rollouts. It demonstrates that training the latent representation only for reconstruction leads to instability, and proposes a set of training interventions—Koopman operator learning, Hamming noise injection, and multi‑step rollout fine‑tuning—that align the latent space with long‑horizon forecasting. These interventions reduce long‑rollout error by about 40 % and achieve accuracy comparable to full‑resolution models while using far fewer floating‑point operations and GPU memory, enabling stable extrapolation in mesoscale crystal‑plasticity simulations of high‑cycle fatigue.
By Andreas E. Robertson, Ashley T. Lenau, John D. Shimanek, Benjamin A. Jasperson, Vivek Oommen, David L. Damm, Krishna Garikipati, Remi Dingreville
arXiv:2505. 11766v4 Announce Type: replace Abstract: Neural Operators (NOs) are powerful architectures for learning mappings between function spaces.
By Haoze Song, Zhihao Li, Xiaobo Zhang, Zecheng Gan, Zhilu Lai, Wei Wang
The paper introduces an in‑span adaptation technique for reduced‑order models, where the reduced subspace is continually updated using the model’s own predictions via an incremental singular‑value decomposition with a forgetting factor. This creates a trajectory‑informed spectral preconditioner that reweights and realigns the basis without changing the subspace, enabling the model to better absorb future out‑of‑span corrections. The authors demonstrate the method on a 3‑D spiral example and nonlinear PDEs such as viscous Burgers and Fisher–KPP, and relate the approach to in‑context learning in dynamical systems.
By Amirpasha Hedayat, Laura Balzano, Karthik Duraisamy
SPARCL introduces a spectral partitioned analytic continual learning method that addresses forgetting in analytic class‑incremental learning. By decomposing the running autocorrelation into a high‑energy core and a residual complement, SPARCL freezes core components for old classes and updates only the residual block, ensuring closed‑form updates with an invariance guarantee. Experiments on CIFAR‑100, CUB‑200, ImageNet‑R, and ImageNet‑A with a frozen ViT‑B/16 protocol show that SPARCL narrows the performance gap between classical analytic learners and strong representation matchers while complementing sparse feature‑decorrelation approaches.
By James Hartley, Zeropy Surio, Daniel Whitmore, Hannah Clarke, Thomas Reed
arXiv:2509. 11285v2 Announce Type: replace-cross Abstract: Class-Incremental Learning (CIL) in deep neural networks is conventionally framed as an iterative gradient-based optimization problem, incurring high computational cost, hyperparameter sensitivity, and risk of catastrophic forgetting.
By Alejandro Dopico-Castro, Oscar Fontenla-Romero, Bertha Guijarro-Berdi\~nas, Amparo Alonso-Betanzos
arXiv:2606. 00132v1 Announce Type: cross Abstract: While finetuning effectively adapts foundation models to specialized downstream tasks, it can degrade nontarget capabilities acquired during pretraining.
By Dongjun Kim, Adrian de Wynter, Huancheng Chen, Heasung Kim, Haris Vikalo
arXiv:2607. 02937v1 Announce Type: new Abstract: Reduced-order models compress high-dimensional dynamics into low-dimensional representations that can be evaluated rapidly, but they lose accuracy when online dynamics drift beyond the training data.
By Amirpasha Hedayat, Laura Balzano, Karthik Duraisamy
The paper introduces Foundation Preserving LoRA (FoLoRA), a forgetting‑aware optimization framework that balances adaptation to downstream tasks with preservation of pretraining capabilities. FoLoRA uses a first‑order preservation condition to define a forgetting penalty based on pretraining‑proxy activations and a task utility from downstream activations, scoring update directions via a generalized Rayleigh quotient. This spectral coordinate system enables gated Adam updates that reduce low‑utility, high‑penalty directions, and the method constructs pretraining proxy calibration data by sampling from the pretrained model. Experiments on math, code, and instruction‑following tasks demonstrate that FoLoRA achieves a stronger balance between target task performance and aggregate preservation of non‑target capabilities compared to baselines.
By Dongjun Kim, Adrian de Wynter, Huancheng Chen, Heasung Kim, Haris Vikalo
arXiv:2609. 19039v1 Announce Type: cross Abstract: We introduce the Long-Short-Range Neural Network (LSR-Net), a novel neural operator architecture designed for data-driven forward evolution modeling, and extends it to the prediction of nonlinear fluid dynamics.
By Qian Hou, Sutrisno, Yuqing Li, Zecheng Gan
arXiv:2602. 20062v2 Announce Type: replace Abstract: Pretraining and fine-tuning are central stages in modern machine learning systems.
By Nicolas Anguita, Francesco Locatello, Andrew M. Saxe, Marco Mondelli, Flavia Mancini, Samuel Lippl, Clementine Domine