arXiv Machine Learning September 7, 2026 By Amar Alem Koric, Qibang Liu, Seid Koric

Disentangling Attention in Deep Operator Learning: A Controlled Study of Data-Driven and Physics-Informed Architectures

Read the original on arXiv Machine Learning →

The paper investigates how different attention mechanisms affect the performance of DeepONet neural operators. Five variants—varying in cross‑attention, self‑attention, tokenization, and attention depth—are trained in both data‑driven and physics‑informed settings on one‑ and two‑dimensional PDE benchmarks. Results show that per‑sensor tokenization with cross‑attention consistently reduces error, while branch self‑attention helps only in complex spatial problems, and deeper cross‑attention yields diminishing returns with higher cost.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 5

When Attention Beats Fourier: Multi-Scale Transformers for PDE Solving on Irregular Domains

arXiv:2605. 08318v2 Announce Type: replace Abstract: We study the problem of \emph{architecture selection} for deep learning models trained to solve partial differential equations (PDEs), asking when transformer-based architectures with learned attention outperform Fourier-domain neural operators.

By Brandon Yee, Pairie Koh, Jack Rodriguez, Mihir Tekal

llms diffusion benchmarks safety

arXiv Machine Learning

Aug 14

Physics-Informed Laplace Neural Operator for Solving Partial Differential Equations

arXiv:2602. 12706v2 Announce Type: replace Abstract: Neural operators have emerged as fast surrogate solvers for parametric partial differential equations (PDEs).

By Heechang Kim, Qianying Cao, Hyomin Shin, Seungchul Lee, George Em Karniadakis, Minseok Choi

rag diffusion benchmarks

arXiv Machine Learning

Jun 5

On the training of physics-informed neural operators for solving parametric partial differential equations

arXiv:2606. 06164v1 Announce Type: new Abstract: Physics-informed neural operators (PINOs) aim to learn solution operators for partial differential equations by using the governing physics as supervision, rather than relying solely on paired input-output simulation data.

By Nanxi Chen, Chuanjie Cui, Airong Chen, Sifan Wang, Rujin Ma

llms computer-vision benchmarks

Hugging Face Trending Papers

Jun 4

On the training of physics-informed neural operators for solving parametric partial differential equations

Physics-informed neural operators (PINOs) aim to learn solution operators for partial differential equations by using the governing physics as supervision, rather than relying solely on paired input-output simulation data. By incorporating physical constraints into the training objective, PINOs combine the cross-instance generalization of neural operators with the data efficiency of physics-informed learning.

llms computer-vision benchmarks

arXiv Machine Learning

Jun 3

Are Deep Learning Based Hybrid PDE Solvers Reliable? Why Training Paradigms and Update Strategies Matter

arXiv:2602. 06842v2 Announce Type: replace-cross Abstract: Deep learning-based hybrid iterative methods (DL-HIMs) integrate classical numerical solvers with neural operators, utilizing their complementary spectral biases to accelerate convergence.

By Yuhan Wu, Jan Willem van Beek, Victorita Dolean, Alexander Heinlein

arXiv AI

Aug 11

Self-Attention to Operator Learning-based 3D-IC Thermal Simulation

arXiv:2510. 15968v2 Announce Type: replace-cross Abstract: Thermal management in 3D ICs is increasingly challenging due to higher power densities.

By Zhen Huang, Hong Wang, Wenkai Yang, Muxi Tang, Depeng Xie, Ting-Jung Lin, Yu Zhang, Wei W. Xing, Lei He

fine-tuning benchmarks

One email a morning, machine-written

Related stories

When Attention Beats Fourier: Multi-Scale Transformers for PDE Solving on Irregular Domains

Physics-Informed Laplace Neural Operator for Solving Partial Differential Equations

On the training of physics-informed neural operators for solving parametric partial differential equations

On the training of physics-informed neural operators for solving parametric partial differential equations

Are Deep Learning Based Hybrid PDE Solvers Reliable? Why Training Paradigms and Update Strategies Matter

Self-Attention to Operator Learning-based 3D-IC Thermal Simulation