The paper investigates how different attention mechanisms affect the performance of DeepONet neural operators. Five variants—varying in cross‑attention, self‑attention, tokenization, and attention depth—are trained in both data‑driven and physics‑informed settings on one‑ and two‑dimensional PDE benchmarks. Results show that per‑sensor tokenization with cross‑attention consistently reduces error, while branch self‑attention helps only in complex spatial problems, and deeper cross‑attention yields diminishing returns with higher cost.
By Amar Alem Koric, Qibang Liu, Seid Koric
arXiv:2512. 09165v2 Announce Type: replace Abstract: Deep Operator Networks (DeepONets) have emerged as a powerful framework for data-driven operator learning, providing flexible surrogates for nonlinear mappings arising in partial differential equations (PDEs).
By Muhammad Abid, Omer San
The paper introduces two multi-stage neural operator learning frameworks—Deep Collocation Neural Operator (DCNO) and Deep Galerkin Neural Operator (DGNO)—for efficiently computing convolution integrals. DCNO is a supervised method that iteratively refines operator approximations by learning residuals from data pairs, while DGNO is an unsupervised approach that uses the weak form of a PDE residual when the operator can be represented by a PDE. Both frameworks build basis operators across multiple training stages, yielding markedly higher accuracy than one-shot learning and achieving near machine‑precision results for convolution problems, with significant efficiency gains for repeated queries or parametric variations.
By Zhiping Mao, Zhenye Wen, Yong Zhang, Xiaofei Zhao
arXiv:2606. 17460v1 Announce Type: new Abstract: Neural operators are widely used as surrogate solution maps for partial differential equations (PDEs), but full-size models can be costly to store, deploy, and evaluate in many-query scientific workflows.
By Lennon J. Shikhman
arXiv:2607. 19378v1 Announce Type: new Abstract: Subquadratic alternatives to attention require compromises when applied to multi-dimensional data: standard convolutions lack global receptive fields and input dependency, while recurrent models require rasterizing data such as images, volumes, and partial differential equation (PDE) into an ad-hoc $1\rm D$ scan order that violates their spatial structure.
By David R. Wessels, Farhad Ramezanghorbani, David W. Romero, Alireza Moradzadeh, Olivia Viessmann, Maksim Zhdanov, John St. John, Ken Janik, David M Knigge, Yucheng Tang, Erik J Bekkers, Saee Gopal Paliwal
arXiv:2505. 11766v4 Announce Type: replace Abstract: Neural Operators (NOs) are powerful architectures for learning mappings between function spaces.
By Haoze Song, Zhihao Li, Xiaobo Zhang, Zecheng Gan, Zhilu Lai, Wei Wang
arXiv:2602. 11626v3 Announce Type: replace-cross Abstract: Learning solution operators on arbitrary geometries remains a central challenge in scientific machine learning, especially for many-query simulation, physics-informed learning, and evolving geometries requiring accurate, geometry-aware predictions at arbitrary spatial locations.
By Wenqian Chen, Zhi-Feng Wei, Yucheng Fu, Michael Penwarden, Pratanu Roy, Panos Stinis
arXiv:2606. 06164v1 Announce Type: new Abstract: Physics-informed neural operators (PINOs) aim to learn solution operators for partial differential equations by using the governing physics as supervision, rather than relying solely on paired input-output simulation data.
By Nanxi Chen, Chuanjie Cui, Airong Chen, Sifan Wang, Rujin Ma
arXiv:2501. 04339v2 Announce Type: replace-cross Abstract: We introduce the Deep Convolutional Interpreter for Time Series (DCIts), a deep-learning architecture for nonlinear multivariate time series that provides sample-specific, locally interpretable descriptions of the underlying interaction structure.
By Domjan Baric, Davor Horvatic
Physics-informed neural operators (PINOs) aim to learn solution operators for partial differential equations by using the governing physics as supervision, rather than relying solely on paired input-output simulation data. By incorporating physical constraints into the training objective, PINOs combine the cross-instance generalization of neural operators with the data efficiency of physics-informed learning.
arXiv:2606. 20035v1 Announce Type: cross Abstract: Many dense prediction networks rely on additive feature transformations and model higher-order feature interactions only implicitly.
By Ziyuan Li, Osamah Sufyan, Uwe Jaekel, Babette Dellen
The paper discusses tensorizing neural networks by reshaping dense weight matrices into higher-order tensors and approximating them with low-rank tensor network decompositions. This approach offers promising model compression and introduces bond indices that create new latent spaces, potentially enhancing interpretability. Despite encouraging empirical results, tensorized neural networks remain underused, and the authors call for more research to address practical scaling and adoption challenges.
By Safa Hamreras, Sukhbinder Singh, Rom\'an Or\'us