The paper reports on a self‑improving science agent that has admitted 252 scientific relations and uses Minimum Description Length (MDL) compression ratios as an unsupervised epistemic grader. The MDL‑ranked top discoveries match canonical physical laws (e.g., vis‑vis‑a, Balmer/Rydberg) with strong statistical enrichment and no counterexamples, while 86 % of the agent’s ledger entries compress poorly (cr≈1) and are identified as statistical regularities rather than true laws. The study introduces a dual‑axis bookkeeping system separating prediction value (R²) from compression value (cr), demonstrates its robustness, and presents falsifiable predictions.
By Wenbin Mao, Pei Zou
arXiv:2607. 05682v1 Announce Type: new Abstract: LLM systems for scientific discovery increasingly assist with ideation, literature synthesis, experiment planning, and report generation, but the first research question they propose can remain difficult to audit: it may sound plausible without exposing the mechanism, falsifier, or assumption that a scientist should inspect.
By Yufeng Wang
The paper presents an explore‑then‑commit protocol that uses a large language model to generate hypotheses, a programmatic planner to collect measurements, and a fresh prompt to synthesize a scientific law from fixed observations. In 576 NewtonBench trials across 12 physics modules, the protocol—especially when interpreter‑enabled planners are used—reduces the number of measurements needed and improves root‑mean‑squared logarithmic error for both GPT‑4.1‑mini and GPT‑4.1. The study demonstrates measurement savings in every module, though it notes that the causal components and generalization beyond noiseless direct‑equation tasks remain unresolved.
By Kautik Mandve, Dileepa Fernando
The paper demonstrates that emergent capabilities in machine learning models can be forecasted with lead time, calibrated uncertainty, and controlled false‑alarm rates. Using per‑seed analysis on transformers, the authors show that the formation time of a previous‑token head predicts the emergence of an induction head with Spearman ρ = 0.977 and a median lead of 975 training steps. Conformal intervals, blind pre‑registered tests, and a multiplicative rule relating anchor and event times further validate the predictive framework across multiple model families and configurations.
By Gunner Levi Howe
arXiv:2607. 00276v1 Announce Type: cross Abstract: Current large-language-model (LLM) physics benchmarks are usually scored by answer accuracy, which cannot distinguish genuine reasoning from recall of familiar problem patterns and reveals little about where a model's reasoning breaks down.
By Dong Zhang
The paper introduces the Infinity Machine, a self‑improving AI agent designed to solve research problems whose solutions can be programmatically verified. The system operates autonomously: it generates candidate solutions, verifies them, and audits the results without human intervention, all within a sandboxed environment governed by a kernel‑enforced gate. In a six‑day experiment, the Infinity Machine achieved four new state‑of‑the‑art results in quantum information and improved qubit‑routing and hardware efficiency, while also documenting several failure modes.
By Harish Santhanalakshmi Ganesan
The paper introduces a framework for evaluating large language model agents by attempting to end‑to‑end reproduce published astronomy studies, separating execution from verification and distinguishing computational failures from methodological ambiguities. Applying this to fourteen papers—one from The Astrophysical Journal and thirteen from Nature—revealed that eleven contained ambiguities that prevented a uniquely specified reproduction path. In a controlled case study, twelve different analysis paths produced distance estimates ranging from 2.16 to 3.53 kpc, with only one matching the published value of ~2.70 kpc, demonstrating that matching outcomes does not guarantee that the agent has reconstructed the underlying reasoning.
whyItMatters":"The study shows that end‑to‑end reproduction can expose gaps in implicit scientific knowledge within AI systems, highlighting the need for better integration of causal relevance in LLM agents."
By Yuehui Wang, Xinyu Qi, Guirong Xue, Cheng Wang, Yangbin Xie, Xiaoyu Tang, Cong Sun
arXiv:2608. 04872v1 Announce Type: cross Abstract: Symbolic regression aims to discover closed-form equations from data, but existing LLM-guided methods often rely on a unified proposal loop that compresses heterogeneous search failures into a scalar score and a single prompt.
By Wenxiao Zhao, Dong Liu, Kaiyi Xu, Feng Liu, Zhen Zhao, Fei Ben, Shu Wang, Wenhao Li, Yingnian Wu, Fenghua Ling, Haobo Li, Lei Bai
arXiv:2608.21356v1 Announce Type: cross
Abstract: For sixty years, machine verification has been a major cost overhead, affordable only for exceptional artifacts. Here we report that generative AI in...
By Jason Hickey
arXiv:2607. 07379v1 Announce Type: new Abstract: In agentic scientific machine learning (SciML), large language model (LLM) agents can discover surrogate models and select one by an automated score, typically an error metric.
By Diab W. Abueidda, Bilal Ahmed, Panos Pantidis, Mostafa E. Mobasher
arXiv:2606. 17529v1 Announce Type: cross Abstract: Scientific machine-learning (SciML) surrogates approximate expensive simulations, but exact expected outputs for arbitrary inputs are unavailable (the oracle problem).
By Meng Li, Xiaohua Yang, Jie Liu, Shiyu Yan
The paper "Beyond Execution: Auditing Experimental Fidelity in LLM-Driven Scientific Research" argues that large language model agents must faithfully implement reference methods, design experiments that truly test claims, and provide supporting evidence. It reports that agents often hallucinate methodology—reducing datasets, substituting components, or drawing conclusions from limited resources—leading to false claims. To counter this, the authors introduce ABE‑Ralph, a reference‑anchored auditing framework that structures experimental constraints, guides implementation, and verifies results, achieving a 93% robust execution rate across 30 reproduction runs and matching or exceeding state‑of‑the‑art performance on 5 NatureBench tasks.
"whyItMatters":"The study demonstrates that evaluating AI scientists requires more than code execution; it must ensure experimental design and evidence truly support the claimed scientific outcomes."
By Lezhi Yu, Xiaogang Xu, Yuhua Zhou, Shuibing He, Aimin Pan