← Back to all news
arXiv Machine Learning By Quan Gu, Xiaoduo Li, Hongxia Liu

Feature Interaction Modeling for Neural Operators

Read the original on arXiv Machine Learning →

The paper introduces the Feature Interaction Modeling Operator (FM-Operator), a point‑wise query neural operator that explicitly designs feature construction and multiplicative interactions between sensor observations and query coordinates. By reinterpreting the DeepONet aggregation as a diagonally constrained multiplicative interaction, FM-Operator restructures both feature construction and interaction to enable richer information exchange while maintaining point‑wise evaluation. Experiments on several PDE benchmarks show that FM-Operator consistently outperforms vanilla DeepONet and improves upon the Shift‑DeepONet baseline, indicating that tailored representation construction and interaction can enhance query‑based neural operator performance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Sep 7

Disentangling Attention in Deep Operator Learning: A Controlled Study of Data-Driven and Physics-Informed Architectures

The paper investigates how different attention mechanisms affect the performance of DeepONet neural operators. Five variants—varying in cross‑attention, self‑attention, tokenization, and attention depth—are trained in both data‑driven and physics‑informed settings on one‑ and two‑dimensional PDE benchmarks. Results show that per‑sensor tokenization with cross‑attention consistently reduces error, while branch self‑attention helps only in complex spatial problems, and deeper cross‑attention yields diminishing returns with higher cost.

By Amar Alem Koric, Qibang Liu, Seid Koric
diffusionnlpbenchmarks
More like this →
arXiv Machine Learning
Aug 20

Multi-stage neural operator learning with application for convolutions

The paper introduces two multi-stage neural operator learning frameworks—Deep Collocation Neural Operator (DCNO) and Deep Galerkin Neural Operator (DGNO)—for efficiently computing convolution integrals. DCNO is a supervised method that iteratively refines operator approximations by learning residuals from data pairs, while DGNO is an unsupervised approach that uses the weak form of a PDE residual when the operator can be represented by a PDE. Both frameworks build basis operators across multiple training stages, yielding markedly higher accuracy than one-shot learning and achieving near machine‑precision results for convolution problems, with significant efficiency gains for repeated queries or parametric variations.

By Zhiping Mao, Zhenye Wen, Yong Zhang, Xiaofei Zhao
More like this →
arXiv Machine Learning
Jul 23

Native Multi-Dimensional Subquadratic Operators via Input Dependent Long Convolutions

arXiv:2607. 19378v1 Announce Type: new Abstract: Subquadratic alternatives to attention require compromises when applied to multi-dimensional data: standard convolutions lack global receptive fields and input dependency, while recurrent models require rasterizing data such as images, volumes, and partial differential equation (PDE) into an ad-hoc $1\rm D$ scan order that violates their spatial structure.

By David R. Wessels, Farhad Ramezanghorbani, David W. Romero, Alireza Moradzadeh, Olivia Viessmann, Maksim Zhdanov, John St. John, Ken Janik, David M Knigge, Yucheng Tang, Erik J Bekkers, Saee Gopal Paliwal
computer-vision
More like this →