arXiv:2606. 19538v1 Announce Type: new Abstract: Convolutional networks, recurrent networks, and transformers each encode different inductive biases -- locality, sequential memory, and content-dependent pairwise interaction -- and have remained mathematically distinct since their inception.
By Ashim Dhor, Rasel Mondal, Pin Yu Chen
arXiv:2610.17424v1 Announce Type: cross
Abstract: Respecting a-priori known symmetries of underlying data is a key principle in designing efficient neural architectures. Convolutional neural networks...
By Tomas Karella, Emily Shinkle, Alice Elisabeth Anastasia Allen, Pieter J. Swart, Nicholas Lubbers, Roxana Bujack
CrossGMN introduces a graph metanetwork that processes a trained source network and an initialized target network simultaneously, enabling equivariant cross‑architecture weight‑space transformations. By preserving symmetry through cross‑network message passing, CrossGMN can refine target network initializations while remaining invariant to source permutations and equivariant to target permutations. Experiments demonstrate that CrossGMN accelerates knowledge distillation, transfers across datasets without retraining, and unifies compression from diverse source architectures into a common target architecture.
By Adir Dayan, Yam Eitan, Haggai Maron
The paper introduces the Feature Interaction Modeling Operator (FM-Operator), a point‑wise query neural operator that explicitly designs feature construction and multiplicative interactions between sensor observations and query coordinates. By reinterpreting the DeepONet aggregation as a diagonally constrained multiplicative interaction, FM-Operator restructures both feature construction and interaction to enable richer information exchange while maintaining point‑wise evaluation. Experiments on several PDE benchmarks show that FM-Operator consistently outperforms vanilla DeepONet and improves upon the Shift‑DeepONet baseline, indicating that tailored representation construction and interaction can enhance query‑based neural operator performance.
By Quan Gu, Xiaoduo Li, Hongxia Liu
The paper introduces the Universal Weight Encoder (UWE), a scalable representation for weight-space learning that uses a typed graph of interfaces and modules to respect parameter symmetries across diverse neural network architectures. UWE combines local equivariant updates with global attention over invariant interface summaries, enabling a single encoder to process multiple supported architectures efficiently. Experiments show UWE matches or surpasses architecture‑specific state‑of‑the‑art methods on standard benchmarks and excels on a large zoo of 1,200 transformers, ranking unseen architectures accurately while using significantly less memory.
By Micha{\l} Kozyra, Michael Bronstein
arXiv:2505.09716v4 Announce Type: replace
Abstract: Out-of-distribution (OOD) generalisation through composition requires a system to discover invariant properties from input-output associations and...
By George Dimitriadis, Spyridon Samothrakis