arXiv Machine Learning

Dynamic Quadtree Tokenization and Transformer for Adaptive Mesh PDE Forecasting

arXiv AI
Aug 17

ArGEnT: Arbitrary Geometry-encoded Transformer for Operator Learning

arXiv:2602. 11626v3 Announce Type: replace-cross Abstract: Learning solution operators on arbitrary geometries remains a central challenge in scientific machine learning, especially for many-query simulation, physics-informed learning, and evolving geometries requiring accurate, geometry-aware predictions at arbitrary spatial locations.

By Wenqian Chen, Zhi-Feng Wei, Yucheng Fu, Michael Penwarden, Pratanu Roy, Panos Stinis
arXiv AI
Aug 28

Successive Capacity Growth: Task-Complexity-Driven Width and Depth Expansion for Vision Transformer Encoders in JEPA World Models

The paper introduces Successive Capacity Growth (SCG), a method for adaptively expanding Vision Transformer encoders in Joint-Embedding Predictive Architectures (JEPAs). SCG starts with a minimal encoder and incrementally increases width or depth based on a task‑agnostic test‑and‑verify mechanism, while a Sketched Isotropic Gaussian Regularizer (SIGReg) keeps learned semantic dimensions independent. Experiments on multi‑object dynamics and 2D navigation tasks show that SCG achieves up to 20.3% better prediction loss than fixed small baselines and 23% better than fixed large models, with far greater parameter efficiency and no false‑positive expansions.

By Frederik Berenz
arXiv Machine Learning
Sep 25

M-plicits: Neural Implicit Surfaces via Nested Multiscale Residuals

M-plicits introduces a multiscale framework for neural implicit surfaces that models a surface as a residual sum of MLPs trained on nested neighborhoods. By localizing supervision to narrow bands around previous zero-level sets, the method achieves robustness to noisy input, avoids costly mesh extraction, and enables a multiscale sphere-tracing algorithm with analytical normal computation. Experiments on Stanford and Thingi32 show superior Chamfer distance and IoU metrics compared to existing methods while using far fewer parameters.

By Vin\'icius da Silva, Isabelle Melo, Matheus Bessa, Guilherme Schardong, Luiz Schirmer, Andr\'e Ara\'ujo, Nuno Gon\c{c}alves, H\'elio Lopes, Alberto Raposo, Luiz Velho, Tiago Novello
arXiv Machine Learning
Oct 2

Function-Space Transformer with Adaptive Anchors

The paper introduces the Function‑Space Transformer (FST), a neural framework that learns from functions by using a spatially adaptive continuous latent representation. FST places features at anchors whose positions are predicted from input samples and refines these anchors through recursive function‑space interactions, allowing the representation to adapt to each input rather than relying on a fixed grid. Experiments on PDEBench (Burgers and Darcy flow) and ImageNet‑1K show that FST outperforms or matches state‑of‑the‑art baselines such as Perceiver IO and the Vision Transformer while using fewer parameters.

By Guorui Sang, Pedram Rooshenas