arXiv AI
Oct 2

ScholarCatalyst: A Benchmark for Retrieving Papers That Inspire New Research

ScholarCatalyst is a new benchmark that evaluates how well AI systems can retrieve research papers that inspire new work. The dataset was created by having 184 lead authors of 207 recent computer science papers annotate which earlier papers helped their projects, providing detailed rationales. The benchmark tests retrieval from the literature available at the start of a project, revealing that current agentic search and even advanced models like Claude Fable 5.1 perform only modestly better than simple embedding retrieval.

By Sohyeon Kim, Yoonho Lee, Bo Liu, Dayoon Ko, Rulin Shao, Seungone Kim, Graham Neubig, Pang Wei Koh, Aakanksha Chowdhery, Akari Asai, Omar Khattab, Yejin Choi, Gunhee Kim, Chelsea Finn
Hugging Face Trending Papers
Jul 29

SciDataSailor: Deep Scientific Data Exploring

Scientific datasets are commonly organized as hierarchical repositories containing heterogeneous and interdependent files, making their inspection, integration, and analysis labor-intensive and reliant on domain expertise. Although large language model (LLM) agents have advanced substantially in planning, reasoning, and tool use, existing research has largely overlooked their ability to interact with real scientific data assets through executable environments.

arXiv AI
Aug 19

AutoResearch: Insight In, Hallucination Out

AutoResearch is a two‑stage autonomous research system that links Idea Generation with Idea Execution. In the generation phase it blends new research signals with existing domain knowledge, identifies transferable mechanistic insights, and produces grounded, testable research plans through multi‑model generation and cross‑review. The execution phase then decomposes these plans into experiments, iteratively implements and diagnoses them, and uses independent evidence‑based review to accept or revise conclusions, thereby turning ideas into measurable progress while minimizing hallucinations.

By Yiming Ren, Xiang Liu, Qumeng Sun, Xiao Zhang, Jiahao Li, Haoyang Zhang, Junjie Wang