Generative AI emulators are increasingly used in scientific domains where we already have strong theory, benchmarks, and physical intuition. This raises a central evaluation and interpretability question: when a foundation-style model can reproduce known continuum dynamics, what internal mechanism supports that behavior, is the internal behaviour consistent with known physics, and how does it relate to where the emulator succeeds or fails?
arXiv:2609.24379v1 Announce Type: cross
Abstract: Mechanistic interpretability of vision transformers seeks to decompose model computation into human-readable units, but learned representations entan...
By Gautam Ranka, Shubham Santosh Pandere, Aiden Dsouza
arXiv:2607. 11958v1 Announce Type: new Abstract: Under the free energy principle, a predictive system does not observe reality directly; it maintains a generative model of the world and experiences that model's best current hypothesis.
By MD Ibrahim Hossain Ridoy
arXiv:2607. 23667v1 Announce Type: cross Abstract: A flow surrogate validated on a simple regime is often taken as evidence that the approach will carry to a richer one.
By Georg Winkler, Martin Stoll
arXiv:2606. 11657v1 Announce Type: cross Abstract: Generative AI emulators are increasingly used in scientific domains where we already have strong theory, benchmarks, and physical intuition.
By Katherine Rosenfeld, Maike Sonnewald
The paper introduces a stability theory for Joint-Embedding Predictive Architectures (JEPAs), showing that training dynamics involve a driving force and a decay effect that determine representation collapse. By linearising the gradient flow, the authors derive a per‑mode stability ratio that separates data‑side and predictor‑side contributions, predicting a phase boundary confirmed across 800 configurations. Using this insight, they propose ResidualPred, a transformer predictor that biases attention toward the identity at initialization, improving representation rank and downstream accuracy on tabular and image benchmarks.
By Jos\'e Lucas De Melo Costa, Seong Woo Ahn, Fabrice Popineau, Arpad Rimmel, Bich-Li\^en Doan
Mechanistic interpretability of vision transformers seeks to decompose model computation into human-readable units, but learned representations entangle many concepts in each neuron. Feature superposi...
arXiv:2604. 00230v2 Announce Type: replace Abstract: Neural collapse (NC) -- the convergence of penultimate-layer features to a simplex equiangular tight frame -- is well understood at equilibrium, but the dynamics governing its onset remain poorly characterised.
By Anamika Paul Rupa
RheOPT is a transformer-based neural operator that first pretrains on inexpensive Newtonian fluid simulations and then fine‑tunes on a small set of non‑Newtonian samples. By reusing geometry‑conditioned transport structures rather than the Newtonian constitutive response, RheOPT achieves accurate few‑shot predictions for complex fluids such as Giesekus, Oldroyd‑B, and Bingham in a contraction channel and for oscillatory wake flows behind a triangular obstacle. The approach demonstrates that Newtonian pretraining can match or surpass models trained from scratch with four times as many non‑Newtonian samples, especially when using reversible instance normalization.
By Maedeh Saberi, Amir Barati Farimani, Safa Jamali
arXiv:2606. 15553v1 Announce Type: cross Abstract: Representation Autoencoders (RAEs) have improved diffusion and flow models by semantically richer latent space owing to the strongly label-wise clustered DINO features in the pretrained encoders.
By Jiawei Zhang, Mengfei Xia, Gen Li, Yuantao Gu
arXiv:2607. 27017v1 Announce Type: new Abstract: A central premise of latent world models is that predicting the future forces a representation to internalize the physics of its environment.
By Kaizhen Tan (New York University, Carnegie Mellon University), Xin Xu (Carnegie Mellon University), Siru Tao (Carnegie Mellon University), Hanzhe Hong (Carnegie Mellon University), Yang Feng (Columbia University), Heqing Du (Columbia University)
LayerRoPE introduces a depth‑wise magnitude and angular superposition mechanism that replaces per‑layer normalization gains with a single shared vector and depth‑conditioned scalars, reducing parameters and computational cost. Across 16 pre‑trained large language models and a 100B+ token model ladder, LayerRoPE outperforms traditional Pre‑, Post‑, and Peri‑Norm schemes, achieving comparable or better performance with significantly less compute and improved convergence up to 512 layers. The method demonstrates that widening the residual stream, rather than shrinking it, enhances depth stability and learning‑rate sensitivity, and it transfers effectively to latent and vision transformer architectures.
By Shikhar Srivastava, Christopher Kanan