The paper investigates how different attention mechanisms affect the performance of DeepONet neural operators. Five variants—varying in cross‑attention, self‑attention, tokenization, and attention depth—are trained in both data‑driven and physics‑informed settings on one‑ and two‑dimensional PDE benchmarks. Results show that per‑sensor tokenization with cross‑attention consistently reduces error, while branch self‑attention helps only in complex spatial problems, and deeper cross‑attention yields diminishing returns with higher cost.
By Amar Alem Koric, Qibang Liu, Seid Koric
HiLNO is a hierarchical latent neural operator that builds a fine‑to‑coarse‑to‑fine latent space and incorporates multi‑scale supervision and anisotropic Gaussian attention to preserve spatial information in PDE solutions with multiscale structures. The hierarchy reduces information loss during compression, while multi‑scale supervision aligns intermediate predictions with downsampled targets, and anisotropic attention facilitates feature transfer across scales. Experiments on standard PDE benchmarks and a large‑scale automotive aerodynamics task show that HiLNO achieves competitive accuracy while cutting parameter count by 84.4% and FLOPs by 69.2% compared with LinearNO, and it generalizes effectively to unseen spatial resolutions.
By Zhicheng Hu, Jiacheng Li, Min Yang
Transolver‑σ is a neural PDE solver that jointly models spectral and physical subspaces to improve accuracy in both one‑step and autoregressive rollouts. The method uses adaptive physical-state interactions, Slice‑Residual Physics‑Attention, and an axis‑factorized Fourier operator to enable information exchange between representations. Across five standard PDE benchmarks, Transolver‑σ reduces benchmark‑averaged relative error by 33.4% compared to the strongest baseline and shows strong performance on coupled multiphysics systems and real‑world fluid and combustion data.
By Haonan Shangguan, Hang Zhou, Haixu Wu, Yuezhou Ma, Jianmin Wang, Mingsheng Long
arXiv:2607. 07718v1 Announce Type: cross Abstract: Neural operators have become a common approach for learning PDE solution maps and accelerating numerical simulations.
By Oded Ovadia, Eli Turkel
arXiv:2602. 11626v3 Announce Type: replace-cross Abstract: Learning solution operators on arbitrary geometries remains a central challenge in scientific machine learning, especially for many-query simulation, physics-informed learning, and evolving geometries requiring accurate, geometry-aware predictions at arbitrary spatial locations.
By Wenqian Chen, Zhi-Feng Wei, Yucheng Fu, Michael Penwarden, Pratanu Roy, Panos Stinis
arXiv:2606. 14934v1 Announce Type: cross Abstract: This work introduces the Separable Neural Architecture (SNA), a function representational class combining neural approximation with tensor decomposition.
By Reza T Batley, Andrew Kichline, Sourav Saha
arXiv:2608.21677v1 Announce Type: cross
Abstract: Recent mesh-based simulation advances have, in no small part, relied on neural surrogates of two distinct families: global models that route informat...
By Anuj Kumar, Heiko Zimmermann, Josiah Bjorgaard, Jacan Chaplais, Nikolaos Bouklas, Matteo Salvador, Alexander Lavin
arXiv:2509.06154v3 Announce Type: replace
Abstract: Developing accurate, data-efficient surrogate models is central to advancing AI for Science. Neural operators (NOs), which approximate mappings bet...
By Dibyajyoti Nayak, Somdatta Goswami
arXiv:2605. 08318v2 Announce Type: replace Abstract: We study the problem of \emph{architecture selection} for deep learning models trained to solve partial differential equations (PDEs), asking when transformer-based architectures with learned attention outperform Fourier-domain neural operators.
By Brandon Yee, Pairie Koh, Jack Rodriguez, Mihir Tekal
arXiv:2608. 09764v1 Announce Type: cross Abstract: Transformer-based neural operators have achieved substantial progress in solving Partial Differential Equations (PDEs) by projecting spatial observations into compact latent tokens and learning physical interactions in latent spaces.
By Zijiang Yang, Xiaomeng Wu, Dongmei Fu
arXiv:2603. 04430v2 Announce Type: replace Abstract: We introduce Flowers, a neural architecture for learning PDE solution operators built entirely from multihead warps.
By Till Muser, Alexandra Spitzer, Matti Lassas, Maarten V. de Hoop, Ivan Dokmani\'c
arXiv:2606. 17460v1 Announce Type: new Abstract: Neural operators are widely used as surrogate solution maps for partial differential equations (PDEs), but full-size models can be costly to store, deploy, and evaluate in many-query scientific workflows.
By Lennon J. Shikhman