arXiv AI

Scientist World: Grow a Research World for AI-Scientists

The paper introduces Scientist World, a platform that transforms research papers into executable environments for AI-Scientists. By extracting open-ended research questions and reproducing proposed methods, the system evaluates AI-Scientists against human authors and has released 1,000 verified environments. The study reports that frontier AI-Scientists surpass human researchers 74% of the time and that training on this platform can significantly boost model performance in research ideation.

arXiv AI
Sep 18

ScientistTwo: Pioneering the Human Knowledge Frontier with Autonomous AI

The paper introduces ScientistTwo, a fully autonomous multi‑agent framework that takes a scientific problem, establishes baselines, generates hypotheses, and coordinates specialized agents to conduct an end‑to‑end discovery cycle without human intervention. It rigorously tests and refines its methods through automated experiments, ablation studies, and a closed‑loop peer‑review engine. Benchmarking against top conferences (ICLR, ICML, NeurIPS) shows that ScientistTwo produces expert‑level, publishable papers and codebases that outperform human state‑of‑the‑art models and receive higher review ratings under automated AI review.

By Jaehyun Nam, Jinsung Yoon, Yanzhou Pan, Yubo Wang, Rui Meng, Parthasarathy Ranganathan, Tomas Pfister
arXiv AI
Sep 25

AI in Science: Early Insights

The paper "AI in Science: Early Insights" analyzes AI’s impact on scientific work using data from 15 million Gemini interactions, 2,600 specialized AI models, and a survey of 600 scientists. It finds widespread AI adoption, complementary use of large language models and specialized tools, significant productivity gains of about seven hours per week, and a shift in research bottlenecks toward hypothesis backlog and verification needs. The study suggests AI can boost scientific productivity but its full effect depends on addressing new downstream challenges.

By Mihai Codreanu, Alex Imas, Juan Mateos-Garcia, Joseph Emmens, Evalyne Muiruri, Arthur Turrell, Julian Jacobs, Atoosa Kasirzadeh, Ana Trisovic, Yiyuan Chen, Tanya Rodchenko, Catherine Pollard, Scott Strand, Daniel Rock, Zanna Iscenko, Fabien Curto Millet, Neil Thompson, James Manyika
arXiv AI
Jun 8

Act As a Real Researcher: A Suite of Benchmarks Evaluating Frontier LLMs and Agentic Harnesses in Research Lifecycle

arXiv:2606. 07462v1 Announce Type: new Abstract: As foundation models advance and agent scaffolding becomes increasingly sophisticated, agents have demonstrated remarkable proficiency in complex, long-horizon coding tasks and even autonomous experiment execution.

By Jiayu Wang, Weijiang Lv, Bowen Fu, Jing Fu, Jiayi Song, Lingyu Zhang, Lanxuan Xue, Luodi Chen, Zepeng Xin, Kaiyu Li, Xiangyong Cao
arXiv AI
Aug 5

Towards a new paradigm of scientific discovery with socialized artificial intelligence

arXiv:2608. 02775v1 Announce Type: new Abstract: Scientific discovery has advanced through successive transformations in the organization of knowledge.

By Xinjie Yao, Xingxin Xu, Xiyuan Gao, Zhoupeng Guo, Kunlong Yang, Dengyu Zhao, Siqi Zhao, Zhihe Fan, Yichen Dong, Xin Li, Jiekang Feng, Jiahe Wu, Sen Wang, Beiming Yu, Kejia Zhao, Ruipu Zhao, Jiaqi Zhou, Heyang Li, Jianjun Chen, Anbo Dai, Xin Liu, Zhengtao Yu, Qinghua Hu, Pengfei Zhu
arXiv Computation and Language
2d ago

IdeaTaste: Do LLMs Share the Research Taste of Human Researchers?

IdeaTaste is a benchmark that evaluates large language models (LLMs) on their ability to generate research proposals that align with what human researchers actually publish. The study compares proposals generated by six contemporary LLMs against real papers from four research areas, finding that LLMs achieve only about 26% of the citation matching and 18% of the idea matching of human communities. Even larger LLM populations and richer field contexts do not close this gap, indicating that LLMs struggle more with translating research directions into concrete approaches than with selecting the right topics.

By Shanshan Zhong, Chen Xu, Jingjie Ning, Zhengzhong Liu, Aviral Kumar, Chenyan Xiong
arXiv Computation and Language
Oct 9

SAIL: Scientific Agentic Intelligence via a Science-Aware Loop

SAIL is an open model with 35 B total and 3 B active parameters designed for literature research, scientific coding, and multi‑step research workflows. It is developed through a science‑aware improvement loop in which agents built on frontier AI models analyze task failures, create training tasks to address capability gaps, and iterate this process across multiple development cycles. The training pipeline combines supervised fine‑tuning, specialist training, multi‑teacher on‑policy distillation, and agentic reinforcement learning, enabling SAIL to achieve competitive performance on scientific research tasks with fewer parameters than leading open‑weight models.

By SAIL Model Team, Boyuan Sun, Bryan Dai, Che Liu, Chi Liu, Derek Li, Hongming Piao, Mengzhuo Chen, Xidong Wang, Yan Shu, Yinda Chen, Ziyang Zeng