arXiv:2509. 13805v4 Announce Type: replace-cross Abstract: Foundation models have revolutionized natural language processing through a ``train once, deploy anywhere'' paradigm, where a single pre-trained model adapts to countless downstream tasks without retraining.
arXiv:2605. 29283v2 Announce Type: replace-cross Abstract: Recent physics foundation models claim general spatiotemporal forecasting ability, yet their evaluations often collapse performance into a single average score under a fixed training distribution.
arXiv:2511. 15684v2 Announce Type: replace-cross Abstract: Foundation models have transformed machine learning for language and vision, but achieving comparable impact in physical simulation remains a challenge.
arXiv:2607. 23377v1 Announce Type: cross Abstract: The largest machine learning models in particle physics are also the most expensive to train, yet the return on scaling a given architecture cannot be estimated before that compute is spent.
The largest machine learning models in particle physics are also the most expensive to train, yet the return on scaling a given architecture cannot be estimated before that compute is spent. Scaling laws have been fit for jets, but none has yet been shown to predict the performance of models it was not fit on.
arXiv:2607. 27501v1 Announce Type: new Abstract: We present a lightweight approach to foundation modeling (\textbf{NEXUS}) that leverages pre-trained learning from collider physics data towards out-of-domain tasks in other scientific datasets, using a fully connected autoencoder model with approximately 3 million parameters.
The paper introduces a framework that separates physical modeling from execution in physics reasoning tasks. It uses a two‑stage post‑training approach: supervised fine‑tuning to build structured models and reinforcement learning with rubric‑based feedback to refine them. Experiments on PhysReason, PhyX, and SeePhys show that this explicit modeling improves reasoning performance by about 3% on average for small LLMs.
arXiv:2503.19081v2 Announce Type: replace Abstract: Scientific foundation models (SciFMs) aim to learn generalizable representations of physical systems governed by partial differential equations (PD...
arXiv:2604. 07366v2 Announce Type: replace Abstract: Partial differential equations (PDEs) govern nearly every physical process in science and engineering, but solving them at scale remains prohibitively expensive.
The paper proposes an unsupervised learning approach that uses program synthesis to translate detailed simulation traces into sparse, high‑level structural patterns, making them easier for large language models to interpret. These pattern detectors can be guided by human‑provided labels such as "rigid collision" or "stretching spring" and produce transparent, explainable functions mapping system states to concise annotations. Experiments on a physics benchmark show that the annotated representations improve natural language reasoning about specific physical systems and enable natural‑language goals to be converted into reward programs for solution search.
arXiv:2607. 24513v1 Announce Type: new Abstract: Transformer architectures have attracted increasing attention for solving partial differential equations (PDEs), owing to their flexibility in handling irregular discretizations and their ability to capture long-range physical dependencies.
arXiv:2605. 17985v2 Announce Type: replace-cross Abstract: We propose a new method for compressing physics foundation models (PFMs) which is a new trend in AI for Science.
The paper investigates whether machine learning models can uncover physical patterns in atomistic data without relying on traditional physics-based inductive biases such as geometric locality or graph structures. By training a general-purpose architecture on molecular simulation data, the authors demonstrate that the model autonomously learns interatomic interaction strengths resembling classical electrostatics and identifies interaction cutoffs aligned with established physical models. The study also reports predictable neural scaling behavior and competitive accuracy on certain metrics compared to physics-informed architectures, suggesting that explicit priors may only be necessary when empirically justified.