← Back to all news
arXiv Machine Learning By Brandon Yee, Pairie Koh, Jack Rodriguez, Mihir Tekal

When Attention Beats Fourier: Multi-Scale Transformers for PDE Solving on Irregular Domains

Read the original on arXiv Machine Learning →

arXiv:2605. 08318v2 Announce Type: replace Abstract: We study the problem of \emph{architecture selection} for deep learning models trained to solve partial differential equations (PDEs), asking when transformer-based architectures with learned attention outperform Fourier-domain neural operators.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Sep 7

Disentangling Attention in Deep Operator Learning: A Controlled Study of Data-Driven and Physics-Informed Architectures

The paper investigates how different attention mechanisms affect the performance of DeepONet neural operators. Five variants—varying in cross‑attention, self‑attention, tokenization, and attention depth—are trained in both data‑driven and physics‑informed settings on one‑ and two‑dimensional PDE benchmarks. Results show that per‑sensor tokenization with cross‑attention consistently reduces error, while branch self‑attention helps only in complex spatial problems, and deeper cross‑attention yields diminishing returns with higher cost.

By Amar Alem Koric, Qibang Liu, Seid Koric
diffusionnlpbenchmarks
More like this →
arXiv Machine Learning
Sep 22

Learning Physics from an Imperfect Ancestor

arXiv:2609.24947v1 Announce Type: new Abstract: Neural operators evaluate parametric partial differential equations cheaply but degrade sharply outside their training distribution. Physics-informed n...

By S. Mohammad Mousavi, Teeratorn Kadeethum, Nikolaos Bouklas, Somdatta Goswami
efficiency
More like this →