arXiv AI
SPEAR: A Spectral-Disentangled MoE Neural Operator with Knowledge-Guided Expert Aggregation for Large-Scale PDE Pretraining
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Sep 10
Latent-MoE: Domain-Aware Mixture-of-Experts for PDEs with Multi-Regime Physics
arXiv:2609.07814v1 Announce Type: new Abstract: Physics-informed neural networks (PINNs) struggle on PDEs whose governing physics varies across the domain. We trace this to a structural property of s...
arXiv Computation and Language
Sep 30
E-MoE: Enhanced Mixture-of-Experts for Non-Factorized Diffusion Language Models
arXiv:2609.37533v1 Announce Type: new Abstract: Masked diffusion models (MDMs) generate sequences by progressively unmasking several tokens per denoising step, but their reverse process is typically...
arXiv AI
Oct 15
SLEM: Spectrum-Aware Latent Expert Merging
arXiv:2610.16484v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) models scale model capacity efficiently by activating only a small subset of experts for each input, while still requiring a...
arXiv Machine Learning
Sep 4
Towards a Statistical Understanding of Mixture-of-Experts
The paper presents a statistical framework for Mixture-of-Experts (MoE) models, treating them as localized aggregation systems. It derives oracle risk bounds that separate approximation, expert‑learning, and router‑estimation errors for both dense and sparse routing with evolving experts. The authors also analyze how sparse Top‑K routing balances computational cost with performance, interpret gating geometrically, and explain how shared experts can capture common predictive structure while allowing routed experts to focus on local residuals.
arXiv Computation and Language
Sep 10
Preserving Long-Tailed Expert Information in Mixture-of-Experts Tuning
arXiv:2604.23036v2 Announce Type: replace-cross Abstract: Despite MoE models leading many benchmarks, supervised fine-tuning (SFT) for the MoE architectures remains difficult because its router layer...
arXiv Machine Learning
Oct 12
PDE-PFN: Prior-Data Fitted Neural PDE Solver
PDE-PFN is a prior‑data fitted neural solver that uses in‑context learning to approximate the posterior predictive mean of partial differential equation (PDE) solutions. It builds on a Transformer‑based PFN architecture with self‑ and cross‑attention, and is pre‑trained on noisy approximate solutions generated by physics‑informed neural networks. Experiments on various two‑dimensional PDEs show that PDE‑PFN generalizes across heterogeneous equations, remains robust to noisy priors, and can perform zero‑gradient‑update inference while achieving competitive or better predictive accuracy compared to baselines.