arXiv AI By Hoda Heidari, Aydin Mohseni

Why and How to Do Science When AI Is Better at It: A Framework Grounded in Human Understanding and Direction

Read the original on arXiv AI →

The paper explores why humans should continue conducting science as AI becomes increasingly capable, arguing that science’s essential goal is durable, collective human understanding of the world for human ends and values. It proposes a two-part test—human understanding and human direction—to decide when research tasks can be delegated to AI, applying this framework across all research phases, including peer review and training. The authors emphasize that this guidance remains relevant even if AI outperforms humans, focusing on the impact of AI on human understanding and accountability rather than on tool type or current AI limitations.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Sep 18

ScientistTwo: Pioneering the Human Knowledge Frontier with Autonomous AI

The paper introduces ScientistTwo, a fully autonomous multi‑agent framework that takes a scientific problem, establishes baselines, generates hypotheses, and coordinates specialized agents to conduct an end‑to‑end discovery cycle without human intervention. It rigorously tests and refines its methods through automated experiments, ablation studies, and a closed‑loop peer‑review engine. Benchmarking against top conferences (ICLR, ICML, NeurIPS) shows that ScientistTwo produces expert‑level, publishable papers and codebases that outperform human state‑of‑the‑art models and receive higher review ratings under automated AI review.

By Jaehyun Nam, Jinsung Yoon, Yanzhou Pan, Yubo Wang, Rui Meng, Parthasarathy Ranganathan, Tomas Pfister
arXiv AI
Aug 19

ASI-Bench: At the Dawn of Artificial Superintelligence

ASI‑Bench is a new benchmark that evaluates AI systems on their ability to conduct innovative exploration and autonomous scientific research across 11 domains, using 60 project‑level tasks. It progressively removes human methodological guidance to test whether AI can independently select methods, execute research, and produce verifiable results. Results from 18 state‑of‑the‑art agent–model configurations show a sharp performance drop when guidance is reduced, indicating current systems still rely heavily on human input.

By Junwei Zhou, Zhen Sun, Binyu Li, Jiangyu Zhou, Yuexi Pan, Hengyu Wang, Honghe Ren, Xiaohan Jia, Xueyang Zhou, Xiaoyu Cao, Yongchao Chen, Yuanning Feng, Junhao Wu, Cheng Zhang, Sijia Chen, Haoyu Xue, Chengsong You, Huan Wang, Koutian Wu, Peigan Gao, Jiakun Wu, Wenzhe Li, Ergan Shang, Qingyuan Zheng, Jingjing Zhou, Ruixuan Jia, Yan Xu, Hongrui Zhang, Xiao-Han Ma, Zhengxiang Cheng, Yuexing Hao, Liting Mai, Xianglin Ji, Wenjun Zhang, Zhuofan Chen, Yixiao Huang, Chi Wang, Wenyue Hua, Yilun Hao, Yuantao Zhai, Ziyan Zhao, Jingyan Xie
arXiv Computation and Language
Nov 4

IdeaTaste: Do LLMs Share the Research Taste of Human Researchers?

IdeaTaste is a benchmark that evaluates large language models (LLMs) on their ability to generate research proposals that align with what human researchers actually publish. The study compares proposals generated by six contemporary LLMs against real papers from four research areas, finding that LLMs achieve only about 26% of the citation matching and 18% of the idea matching of human communities. Even larger LLM populations and richer field contexts do not close this gap, indicating that LLMs struggle more with translating research directions into concrete approaches than with selecting the right topics.

By Shanshan Zhong, Chen Xu, Jingjie Ning, Zhengzhong Liu, Aviral Kumar, Chenyan Xiong
arXiv AI
Oct 27

Closing the empirical loop: autonomous AI agents conduct end-to-end research with human participants

The paper reports that a domain‑agnostic, agentic AI system can autonomously carry out the full scientific workflow—from hypothesis generation to data collection and manuscript preparation—by conducting three online psychological studies involving 288 participants. The AI designed experiments on visual working memory, mental rotation, and imagery vividness, coded analysis pipelines in over eight hours, and produced completed manuscripts, achieving methodological rigor comparable to experienced researchers. The work demonstrates the feasibility of AI‑driven scientific discovery while highlighting limitations in conceptual nuance and raising questions about scientific credit and understanding.

By Gabrielle Wehr, Reuben Rideaux, Amaya J. Fox, David R. Lightfoot, Jason Tangen, Jason B. Mattingley, Shane E. Ehrhardt