The paper investigates whether large language models can autonomously discover altered physical laws in simulated environments. Using the LongChang framework, 12 LLMs explore worlds with modified physics, manipulate objects, and build tools without external guidance. Only GPT‑6‑Astra achieved a 16.7% discovery rate, while the others found none, revealing that current models struggle to fully identify hidden laws.
arXiv:2605.26087v2 Announce Type: replace-cross
Abstract: Frontier LLMs now perform strongly across a wide range of physics evaluations, but it is hard to disentangle genuine reasoning from recall of...
By Matt L. Wiemann, Lindsay M. Smith, Peter Melchior, Siddharth Mishra-Sharma, Andrew Gordon Wilson, Pavel Izmailov, Carolina Cuesta-L\'azaro
The paper presents an explore‑then‑commit protocol that uses a large language model to generate hypotheses, a programmatic planner to collect measurements, and a fresh prompt to synthesize a scientific law from fixed observations. In 576 NewtonBench trials across 12 physics modules, the protocol—especially when interpreter‑enabled planners are used—reduces the number of measurements needed and improves root‑mean‑squared logarithmic error for both GPT‑4.1‑mini and GPT‑4.1. The study demonstrates measurement savings in every module, though it notes that the causal components and generalization beyond noiseless direct‑equation tasks remain unresolved.
By Kautik Mandve, Dileepa Fernando
arXiv:2607. 05682v1 Announce Type: new Abstract: LLM systems for scientific discovery increasingly assist with ideation, literature synthesis, experiment planning, and report generation, but the first research question they propose can remain difficult to audit: it may sound plausible without exposing the mechanism, falsifier, or assumption that a scientist should inspect.
By Yufeng Wang
arXiv:2607. 00276v1 Announce Type: cross Abstract: Current large-language-model (LLM) physics benchmarks are usually scored by answer accuracy, which cannot distinguish genuine reasoning from recall of familiar problem patterns and reveals little about where a model's reasoning breaks down.
By Dong Zhang
arXiv:2609.01526v1 Announce Type: new
Abstract: Scientific agents must learn not only how to reason, but also what to believe. However, existing LLM agents typically express scientific hypotheses in...
By Qing Zhao, Haowei Li, Weijian Deng, Pengxu Wei, Liang Lin