arXiv:2512. 04149v2 Announce Type: replace-cross Abstract: Next token prediction is an attractive pre-training task for jet foundation models, in that it is simulation free and enables excellent generative capabilities that can transfer across datasets.
arXiv:2606. 14870v1 Announce Type: cross Abstract: Foundation models (FMs) trained on large datasets and fine-tuned on downstream tasks have emerged as a powerful paradigm in AI for science.
arXiv:2606. 14813v1 Announce Type: cross Abstract: Jet tagging at the Large Hadron Collider increasingly relies on deep learning models trained on massive simulated datasets, leading to high computational costs and limited robustness to detector mismodeling.
arXiv:2606. 19781v1 Announce Type: cross Abstract: Neural scaling laws describe how model performance improves as a power law in compute, model size, and dataset size.
The paper investigates whether two particle physics foundation models, OmniLearned and ParticleViT, pretrained on high‑energy proton–proton and electron–proton collisions can transfer knowledge to a low‑energy neutrino experiment. Using MINERvA neutrino–nucleus scattering data, the authors evaluate the models on energy regression and charged‑current pion classification tasks, finding that the pretrained models outperform similarly sized models trained from scratch, with OmniLearned excelling in regression and ParticleViT in classification. When the same transformer architecture is initialized from unrelated text pretraining (BERT), the performance advantage is minimal for classification and absent for regression, indicating that particle‑level foundation models capture inductive biases that generalize across energy scales, detector technologies, and physics processes.
arXiv:2606. 14373v1 Announce Type: cross Abstract: The workflow from particle collision to physics analysis passes through a series of reconstruction steps that are traditionally modular and disconnected, with no shared representation linking low-level detector data to high-level analysis tasks.
arXiv:2607. 23377v1 Announce Type: cross Abstract: The largest machine learning models in particle physics are also the most expensive to train, yet the return on scaling a given architecture cannot be estimated before that compute is spent.
The paper introduces a model calibration method using optimal transport to address discrepancies between simulation and experimental data in high-dimensional machine learning applications. Applied to jet tagging in particle physics, the technique calibrates a 128‑dimensional latent representation from a general‑purpose classifier, ensuring downstream derived quantities are properly calibrated. This enables more reliable use of foundation models for jet flavor analysis in LHC experiments and offers a general framework for correcting high‑dimensional simulations across scientific fields.
arXiv:2601.11719v4 Announce Type: replace Abstract: Self-supervised learning, in the context of foundation model training, is a powerful pre-training method for learning feature representations witho...
The largest machine learning models in particle physics are also the most expensive to train, yet the return on scaling a given architecture cannot be estimated before that compute is spent. Scaling laws have been fit for jets, but none has yet been shown to predict the performance of models it was not fit on.
arXiv:2606. 09861v1 Announce Type: cross Abstract: While Next-Token Prediction (NTP) has unified LLM pretraining, its adaptation to unbounded, continuous time series (TS) remains open.
arXiv:2608. 19529v1 Announce Type: cross Abstract: Many real-world AI systems represent entities, behaviors, and structured information using discrete machine-native symbols rather than natural language.
The paper introduces a particle‑level generative model that uses residual‑quantized full‑event data to enable fast, ML‑based surrogate simulation for collider events. It demonstrates conditional generation from detector‑stable particles, explores scaling across dataset and model sizes, and shows that token‑level loss predicts downstream physical fidelity. The work offers an empirical framework for scalable collider full‑event generation using residual‑quantized representations.