← Back to all news
arXiv Machine Learning By Sy-Tuyen Ho, Minghui Liu, Huy Nghiem, Furong Huang

SoundnessBench: Can Your AI Scientist Really Tell Good Research Ideas from Bad Ones?

Read the original on arXiv Machine Learning →

SoundnessBench is a benchmark of 1,099 real research proposals designed to test whether large language models can judge the methodological viability of a research idea before resources are invested. The study evaluates 12 frontier LLMs and finds a pervasive optimism bias, with standard prompting yielding a 74.0% false‑positive rate and aggressive prompting increasing false rejections. Analyses show that this bias persists even when controlling for public corpus contamination, surface features, and literature grounding, indicating that current LLMs are not yet reliable gatekeepers for scientific rigor.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Jun 11

Can AI Agents Synthesize Scientific Conclusions?

arXiv:2606. 11337v1 Announce Type: new Abstract: Scientific AI agents increasingly retrieve evidence, reason across sources, and synthesize conclusions used in consequential decisions.

By Hayoung Jung, Pedro Viana Diniz, Jos\'e Reinaldo Corr\^ea Roveda, Abner Fernandes da Silva, Haeun Jung, Enoch Tsai, Aleksandra Korolova, Manoel Horta Ribeiro
agentsbenchmarks
More like this →
arXiv Computation and Language
2d ago

IdeaTaste: Do LLMs Share the Research Taste of Human Researchers?

IdeaTaste is a benchmark that evaluates large language models (LLMs) on their ability to generate research proposals that align with what human researchers actually publish. The study compares proposals generated by six contemporary LLMs against real papers from four research areas, finding that LLMs achieve only about 26% of the citation matching and 18% of the idea matching of human communities. Even larger LLM populations and richer field contexts do not close this gap, indicating that LLMs struggle more with translating research directions into concrete approaches than with selecting the right topics.

By Shanshan Zhong, Chen Xu, Jingjie Ning, Zhengzhong Liu, Aviral Kumar, Chenyan Xiong
llmsagentsbenchmarks
More like this →
arXiv AI
Aug 19

Do LLMs Know a Good Hypothesis When They See One? Logit-Based Energy Scoring Outperforms Prompted LLM-as-Judge for Scientific Hypothesis Ranking

The paper investigates whether large language models (LLMs) can reliably assess scientific hypotheses by using a logit-based energy scoring method that leverages the model’s intrinsic confidence. Across 1,323 papers in 12 disciplines, this intrinsic scoring achieved a 33.0% Hit@1 rate, outperforming a prompted listwise ranking approach that scored 16.6%. The best result, a 1‑billion‑parameter model with logit-based energy scoring, reached 53.1% Hit@1, suggesting that confidence‑based evaluation could improve trustworthy AI‑enabled scientific discovery.

By Swati Rajwal, Sanjay Das, Tirthankar Ghosal
llms
More like this →