The paper investigates how different attention mechanisms affect the performance of DeepONet neural operators. Five variants—varying in cross‑attention, self‑attention, tokenization, and attention depth—are trained in both data‑driven and physics‑informed settings on one‑ and two‑dimensional PDE benchmarks. Results show that per‑sensor tokenization with cross‑attention consistently reduces error, while branch self‑attention helps only in complex spatial problems, and deeper cross‑attention yields diminishing returns with higher cost.
arXiv:2605. 08318v2 Announce Type: replace Abstract: We study the problem of \emph{architecture selection} for deep learning models trained to solve partial differential equations (PDEs), asking when transformer-based architectures with learned attention outperform Fourier-domain neural operators.
arXiv:2602. 12706v2 Announce Type: replace Abstract: Neural operators have emerged as fast surrogate solvers for parametric partial differential equations (PDEs).
The paper introduces the Cluster Attention Neural Operator (CANO), a neural operator that uses a novel cross‑attention mechanism to dynamically cluster queries while keeping full‑resolution keys and values. This design eliminates the slice compression and weight‑sharing limitations of previous Transformer‑based operators, maintaining fast computation and global interactions. Experiments on a range of fluid and solid dynamics benchmarks—including Navier‑Stokes, Airfoil, Plasticity, Pipe Turbulence, and Composites—show that CANO achieves lower errors than existing baselines and demonstrates strong geometric adaptability and temporal consistency.
The paper introduces a method to amortize physics-informed neural networks (PINNs) across related partial differential equations (PDEs) by explicitly modeling equation relationships in an operator graph. Coefficient vectors encode numerical parameters, while the graph hypernetwork generates diagonal codes that initialize a meta‑trained factorized PINN for each target equation. Experiments on scalar convection‑diffusion‑reaction, two‑field Fisher‑KPP, and a capacitively coupled plasma model show that term‑based descriptors and graph conditioning improve solution accuracy compared to coefficient‑vector conditioning, especially for high‑reaction and coupled systems.
arXiv:2606. 06164v1 Announce Type: new Abstract: Physics-informed neural operators (PINOs) aim to learn solution operators for partial differential equations by using the governing physics as supervision, rather than relying solely on paired input-output simulation data.
Physics-informed neural operators (PINOs) aim to learn solution operators for partial differential equations by using the governing physics as supervision, rather than relying solely on paired input-output simulation data. By incorporating physical constraints into the training objective, PINOs combine the cross-instance generalization of neural operators with the data efficiency of physics-informed learning.
The paper proposes a method to amortize physics-informed neural networks (PINNs) across related partial differential equations (PDEs) by explicitly modeling equation relationships in an operator graph. Coefficient vectors encode numerical parameters, while the graph hypernetwork generates diagonal codes that initialize a meta‑trained factorized PINN for each target equation. Experiments on scalar convection‑diffusion‑reaction, two‑field Fisher‑KPP, and a capacitively coupled plasma model show that term‑based descriptors and graph conditioning improve solution accuracy compared to coefficient‑vector conditioning alone.
Transolver‑σ is a neural PDE solver that jointly models spectral and physical subspaces to improve accuracy in both one‑step and autoregressive rollouts. The method uses adaptive physical-state interactions, Slice‑Residual Physics‑Attention, and an axis‑factorized Fourier operator to enable information exchange between representations. Across five standard PDE benchmarks, Transolver‑σ reduces benchmark‑averaged relative error by 33.4% compared to the strongest baseline and shows strong performance on coupled multiphysics systems and real‑world fluid and combustion data.
The paper introduces a framework for evaluating how well learned states from neural PDE solvers can be reused in new contexts. By separating solution accuracy, learning contribution, and numerical utility through paired state comparisons and cost accounting, the authors audit 12 papers and conduct experiments on linear PDE families, Fourier neural operators, and convolutional networks. Their results show that matched state comparisons and downstream computational evidence can lead to significant savings in conjugate‑gradient iterations and demonstrate finite‑budget pretraining value.
Physics-informed neural PDE solvers typically keep a fixed representation while adapting parameters, which limits performance on heterogeneous problems. The authors propose physics-informed neural plasticity, allowing the representation itself to reshape during training. Their ReCAP method dynamically redistributes capacity via local enrichment, residual-directed splitting, gate-based pruning, and function-aware merging, achieving the lowest relative $L^2$ error on five challenging 3D and 4D benchmarks and reducing error by 10.7%–27.5% compared to the best competing solvers.
The paper introduces PAM-GS, a physics-aware gradient surgery technique for Physics-Informed Neural Networks (PINNs). It addresses the highly imbalanced multi-task optimisation problem in PINNs by adaptively mitigating task interference based on observed gradient conflicts. Experiments on four PDE benchmarks show that PAM-GS achieves competitive solution accuracy while maintaining strong task-balanced performance, outperforming existing methods on most problems.
arXiv:2602. 06842v2 Announce Type: replace-cross Abstract: Deep learning-based hybrid iterative methods (DL-HIMs) integrate classical numerical solvers with neural operators, utilizing their complementary spectral biases to accelerate convergence.