arXiv:2609.07061v2 Announce Type: replace Abstract: Physics-Informed Neural Networks (PINNs) embed PDE residuals into neural network training, but their internal representations remain opaque: it is...
arXiv:2607. 12166v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) are the standard for decomposing superposed neural representations into interpretable features, and evaluation relies predominantly on correlational recovery metrics -- cosine similarity between ground-truth directions and decoder atoms.
arXiv:2607. 01336v1 Announce Type: cross Abstract: Neural Quantum States (NQS) are a remarkably expressive class of variational ans\"atze for quantum many-body wavefunctions, yet little is understood about their internal mechanisms: trained on variational objectives alone, how do NQS accurately capture physical observables that they have never been explicitly optimized for?
The paper applies sparse autoencoders to a neutrino foundation model trained on IceCube data, uncovering a validated atlas of physical concepts within the model’s internal representation. Causal analysis shows the direction reconstruction head largely ignores this atlas, whereas an uncertainty head trained on the same representation effectively uses quality and brightness features, improving angular resolution from 20.2° to 3.2° at 20% efficiency. These findings demonstrate that mechanistic interpretability can expose latent physics and guide the design of downstream tasks.
arXiv:2606. 11657v1 Announce Type: cross Abstract: Generative AI emulators are increasingly used in scientific domains where we already have strong theory, benchmarks, and physical intuition.
arXiv:2607. 07127v1 Announce Type: cross Abstract: Lattice field theory is the workhorse of non-perturbative physics, used to simulate phenomena from the strong nuclear force to critical phenomena in materials.
Physics-Informed Neural Networks (PINNs) provide a meshless approach for solving partial differential equations (PDEs), but suffer severe degradation in stiff and shock-dominated problems, where small PDE residuals can correspond to globally inaccurate solutions. We show these failures are multi-causal, arising from the concurrent interplay of (i) spectral bias against sharp features, (ii) imbalanced multi-term optimization and loss-weight collapse, (iii) violation of temporal causality, and (iv) under-resolved collocation.
Generative AI emulators are increasingly used in scientific domains where we already have strong theory, benchmarks, and physical intuition. This raises a central evaluation and interpretability question: when a foundation-style model can reproduce known continuum dynamics, what internal mechanism supports that behavior, is the internal behaviour consistent with known physics, and how does it relate to where the emulator succeeds or fails?
The paper reports three pre‑registered studies on 2D Navier‑Stokes flow that examine whether self‑supervised models organize their latent spaces around physical fluid quantities. In Study 1, a joint‑embedding predictive architecture (JEPA) fails to outperform an untrained network in capturing fluid dynamics, while a JEPA trained on non‑physical dynamics and a static invariance model perform equally well or better. Study 2 shows that plain autoencoders make enstrophy prominent in their slowest latent directions, and adding VICReg reduces this prominence but improves linear decodability. Study 3 constructs steerable latent directions (“dials”) via routing; a loudness dial is found in all eight trained models and a shape dial in three of five seeds, though neither control shows a shape dial. Across all studies, the content of a latent and what can be controlled through it diverge, and exploratory experiments reveal that supervised spectral‑shape directions do not affect the output when steered.
MOSAIC is a sparse temporal variational autoencoder that merges causal representation learning with support recovery to identify interpretable latent modules in scientific time series. It leverages regime‑conditioned temporal variation to discover latent variables and uses an additive decoder to recover a sparse set of observations linked to each latent, providing module‑level interpretability. The method offers theoretical guarantees for ANOVA main‑effect identifiability under smooth mixing functions and demonstrates empirical success across diverse domains such as RNA dynamics, solar wind, ENSO climate, industrial process control, and synthetic tokamak data.
arXiv:2503.19081v2 Announce Type: replace Abstract: Scientific foundation models (SciFMs) aim to learn generalizable representations of physical systems governed by partial differential equations (PD...
arXiv:2606. 02385v1 Announce Type: cross Abstract: Sparse Autoencoders (SAEs) have found success parsing neural representations into interpretable concepts, providing a basis for understanding and control.