arXiv AI By Yiming Huang, Ziche Liu, Zhuohang Wu, Yiqian Wang, Junxia Cui, Xinkai Zou, Linjun Mao, Nan Huang, Naicheng Yu, Kaijie Zhu, Yue Ma, Kun Zhou, Letian Peng, Jingbo Shang

Can LLMs Discover Scientific Laws in Real and Parallel Worlds?

Read the original on arXiv AI →

The paper introduces SCILAWS-BENCH, a benchmark for evaluating large language models (LLMs) on scientific law discovery. It contains 118 problems from 381 scientific papers, covering 291 candidate laws and about 8 million real data points across six disciplines. The benchmark offers two settings: SCILAWS-REAL, where models must propose laws from fixed real observations, and SCILAWS-PARALLEL, where models actively query synthetic worlds to recover hidden laws.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Jul 14

FIRE-Bench: Evaluating AI Agents on the Rediscovery of Scientific Insights

arXiv:2602. 02905v2 Announce Type: replace Abstract: Autonomous agents powered by large language models (LLMs) promise to accelerate scientific discovery end-to-end, but rigorously evaluating their capacity for verifiable discovery remains a central challenge.

By Zhen Wang, Fan Bai, Zhongyan Luo, Jinyan Su, Kaiser Sun, Xinle Yu, Jieyuan Liu, Kun Zhou, Claire Cardie, Mark Dredze, Zhiting Hu, Eric P. Xing
arXiv AI
Aug 19

SGHA: Evidence-Grounded Research Problem Discovery with Local Language Models

The paper introduces SGHA, a fully automated system that discovers research problems by structuring scientific literature into evidence-linked objects and a typed evidence graph. SGHA operates entirely on a local 9B open‑weight language model, avoiding proprietary frontier‑model APIs, and outputs traceable research‑problem families with assumptions, objectives, success criteria, and ambiguities. Comparative experiments in five machine‑learning domains show that SGHA’s corpus‑first, evidence‑constrained approach yields inspectable research‑problem formulation without relying on external models.

By Sarvesh Gharat, Junpei Komiyama
arXiv AI
Sep 25

AI in Science: Early Insights

The paper "AI in Science: Early Insights" analyzes AI’s impact on scientific work using data from 15 million Gemini interactions, 2,600 specialized AI models, and a survey of 600 scientists. It finds widespread AI adoption, complementary use of large language models and specialized tools, significant productivity gains of about seven hours per week, and a shift in research bottlenecks toward hypothesis backlog and verification needs. The study suggests AI can boost scientific productivity but its full effect depends on addressing new downstream challenges.

By Mihai Codreanu, Alex Imas, Juan Mateos-Garcia, Joseph Emmens, Evalyne Muiruri, Arthur Turrell, Julian Jacobs, Atoosa Kasirzadeh, Ana Trisovic, Yiyuan Chen, Tanya Rodchenko, Catherine Pollard, Scott Strand, Daniel Rock, Zanna Iscenko, Fabien Curto Millet, Neil Thompson, James Manyika