The paper introduces the Cluster Attention Neural Operator (CANO), a neural operator that uses a novel cross‑attention mechanism to dynamically cluster queries while keeping full‑resolution keys and values. This design eliminates the slice compression and weight‑sharing limitations of previous Transformer‑based operators, maintaining fast computation and global interactions. Experiments on a range of fluid and solid dynamics benchmarks—including Navier‑Stokes, Airfoil, Plasticity, Pipe Turbulence, and Composites—show that CANO achieves lower errors than existing baselines and demonstrates strong geometric adaptability and temporal consistency.
By Ming Zhong, Antonio Colanera, Gianluigi Rozza, Zhenya Yan
arXiv:2608. 09764v1 Announce Type: cross Abstract: Transformer-based neural operators have achieved substantial progress in solving Partial Differential Equations (PDEs) by projecting spatial observations into compact latent tokens and learning physical interactions in latent spaces.
By Zijiang Yang, Xiaomeng Wu, Dongmei Fu
arXiv:2606. 14597v1 Announce Type: new Abstract: Transformer-based neural operators have shown remarkable performance for approximating solution operators of partial differential equations on complex geometries.
By Armand de Villeroch\'e, Sibo Cheng, Vincent Le Guen, Marc Bocquet, Rem-Sophia Mouradi, Patrick Armand, Alban Farchi, Patrick Massin
arXiv:2608. 07053v1 Announce Type: new Abstract: Pretrained partial differential equation (PDE) foundation models can generalize across different equations, but adapting them to unseen PDE systems typically requires dense solution data, which is often expensive or unavailable.
By Ziye Song, Zhao Wei, Xin Yu, Ivor Tsang, Yueming Lyu
arXiv:2611.05735v1 Announce Type: new
Abstract: Neural operators fit their representation of functions to the task they are trained on, so every new task rebuilds a reduced space from labeled data. F...
By Sepehr Mousavi, Levi Lingsch, Siddhartha Mishra, Laura De Lorenzis
arXiv:2603. 19312v3 Announce Type: replace Abstract: Joint Embedding Predictive Architectures (JEPAs) offer a compelling framework for learning world models in compact latent spaces, yet existing methods remain fragile, relying on complex multi-term losses, exponential moving averages, pre-trained encoders, or auxiliary supervision to avoid representation collapse.
By Lucas Maes, Quentin Le Lidec, Damien Scieur, Yann LeCun, Randall Balestriero
arXiv:2607. 07718v1 Announce Type: cross Abstract: Neural operators have become a common approach for learning PDE solution maps and accelerating numerical simulations.
By Oded Ovadia, Eli Turkel
The paper investigates how different attention mechanisms affect the performance of DeepONet neural operators. Five variants—varying in cross‑attention, self‑attention, tokenization, and attention depth—are trained in both data‑driven and physics‑informed settings on one‑ and two‑dimensional PDE benchmarks. Results show that per‑sensor tokenization with cross‑attention consistently reduces error, while branch self‑attention helps only in complex spatial problems, and deeper cross‑attention yields diminishing returns with higher cost.
By Amar Alem Koric, Qibang Liu, Seid Koric
arXiv:2608.21677v1 Announce Type: cross
Abstract: Recent mesh-based simulation advances have, in no small part, relied on neural surrogates of two distinct families: global models that route informat...
By Anuj Kumar, Heiko Zimmermann, Josiah Bjorgaard, Jacan Chaplais, Nikolaos Bouklas, Matteo Salvador, Alexander Lavin
The paper introduces the Function‑Space Transformer (FST), a neural framework that learns from functions by using a spatially adaptive continuous latent representation. FST places features at anchors whose positions are predicted from input samples and refines these anchors through recursive function‑space interactions, allowing the representation to adapt to each input rather than relying on a fixed grid. Experiments on PDEBench (Burgers and Darcy flow) and ImageNet‑1K show that FST outperforms or matches state‑of‑the‑art baselines such as Perceiver IO and the Vision Transformer while using fewer parameters.
By Guorui Sang, Pedram Rooshenas
The paper introduces Hypernetwork-Parameterized Spatially Adaptive Neural Operators (SANO) for learning partial differential equations (PDEs) with spatial heterogeneity. SANO replaces spatially shared parameterization with a continuous field of location-dependent operator parameters, generated by a coordinate-conditioned hypernetwork and interpolated via a Hyper-Neural Element mechanism. Experiments on 1‑, 2‑, and 3‑dimensional PDEs and perforated-domain elliptic benchmarks demonstrate that SANO consistently outperforms existing neural‑operator, hypernetwork‑based, and physics‑informed baselines.
By Jiaquan Zhang, Chaoning Zhang, Shuxu Chen, Meng Ye, Xiaofeng Zhang, Qiang He, Weifeng Huang, Guoqing Wang, Yang Yang, Caiyan Qin
arXiv:2606. 11963v1 Announce Type: new Abstract: Neural operators provide a powerful framework for learning solution mappings of partial differential equations directly in function space.
By Mostafa Bamdad, Mohammad Sadegh Eshaghi, Timon Rabczuk