arXiv Machine Learning

Unknown Unknowns: Model Misspecification in Machine Learning for Physics

arXiv:2608. 13633v1 Announce Type: cross Abstract: Machine learning is now a central tool for solving inverse problems in particle physics and astronomy.

arXiv Machine Learning
Sep 10

Mind the Gap: Navigating Inference with Optimal Transport Maps

The paper introduces a model calibration method using optimal transport to address discrepancies between simulation and experimental data in high-dimensional machine learning applications. Applied to jet tagging in particle physics, the technique calibrates a 128‑dimensional latent representation from a general‑purpose classifier, ensuring downstream derived quantities are properly calibrated. This enables more reliable use of foundation models for jet flavor analysis in LHC experiments and offers a general framework for correcting high‑dimensional simulations across scientific fields.

By Malte Algren, Tobias Golling, Francesco Armando Di Bello, Christopher Pollard
arXiv AI
4d ago

Reconstructing Implicit Scientific Knowledge: Evaluating LLM Agents through End-to-End Reproduction of Astronomy

The paper introduces a framework for evaluating large language model agents by attempting to end‑to‑end reproduce published astronomy studies, separating execution from verification and distinguishing computational failures from methodological ambiguities. Applying this to fourteen papers—one from The Astrophysical Journal and thirteen from Nature—revealed that eleven contained ambiguities that prevented a uniquely specified reproduction path. In a controlled case study, twelve different analysis paths produced distance estimates ranging from 2.16 to 3.53 kpc, with only one matching the published value of ~2.70 kpc, demonstrating that matching outcomes does not guarantee that the agent has reconstructed the underlying reasoning. whyItMatters":"The study shows that end‑to‑end reproduction can expose gaps in implicit scientific knowledge within AI systems, highlighting the need for better integration of causal relevance in LLM agents."

By Yuehui Wang, Xinyu Qi, Guirong Xue, Cheng Wang, Yangbin Xie, Xiaoyu Tang, Cong Sun