LLM-IDEA is an identifiability‑driven experimental agent that guides large language model scientists in autonomous discovery of mechanistic world models. It evaluates whether a plateau in learning is due to the agent’s capability limits, the model’s resolvability within the design class, or a certified exhaustion of the experiment space. The system demonstrates its effectiveness across benchmarks such as ODEBench, DiscoverPhysics, and a custom Alien Universe testbed, consistently identifying when additional experiments are futile and when new experimental designs are required.
Predicting the answer to interventional ``what if'' questions --- the outcome of an action never taken --- requires a \emph{mechanistic}, causal model, not a curve fit; and learning such a model requires \emph{experiments}, because passive data leaves its mechanisms unidentified. Experiments are expensive, so the central problem is \emph{data efficiency}.
arXiv:2608. 09696v1 Announce Type: new Abstract: Predicting the answer to interventional ``what if'' questions --- the outcome of an action never taken --- requires a \emph{mechanistic}, causal model, not a curve fit; and learning such a model requires \emph{experiments}, because passive data leaves its mechanisms unidentified.
By Kevin Murphy
arXiv:2609.07611v1 Announce Type: new
Abstract: Scientific ideation is the capacity to formulate novel and testable hypotheses from scientific evidence, and autonomous AI scientists depend on it. Exi...
By Yunxiang Mo, Tianshi Zheng, Yisen Gao, Rui Wang, Newt Nguyen Kim Hue Nam, Kelvin Kiu Wai Tam, Jiaxin Bai, Yangqiu Song, Ginny Wong, Simon See
arXiv:2605.26087v2 Announce Type: replace-cross
Abstract: Frontier LLMs now perform strongly across a wide range of physics evaluations, but it is hard to disentangle genuine reasoning from recall of...
By Matt L. Wiemann, Lindsay M. Smith, Peter Melchior, Siddharth Mishra-Sharma, Andrew Gordon Wilson, Pavel Izmailov, Carolina Cuesta-L\'azaro
ULTRADISCOVERY is a benchmark that tests abductive exploration in an epistemically open, interconnected universe. It features five domains where agents must revise an initial theory and predict outcomes of unseen cross‑domain interventions, with a $2 imes 2$ design manipulating representation openness and evidence alignment. Experiments show that agents struggle to compose evidence into transferable representations, especially when the representation is open, and that even with aids exact predictions are rare without large budgets.
By Weihan Li, Tianshi Zheng, Yangqiu Song, Ginny Y. Wong, Simon See