arXiv Machine Learning
The Agentic Scientist: Orchestrating Coding Agents for Automated Scientific (Co-)Discovery
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Sep 3
Can Coding Agents Reproduce Findings in Computational Materials Science?
The paper introduces AutoMat, a benchmark designed to test large language model (LLM) coding agents on their ability to reproduce claims from computational materials science. AutoMat presents three challenges: reconstructing underspecified procedures, navigating specialized toolchains, and assessing whether the evidence supports a claim. Experiments show that current LLM agents achieve low success rates, with the best setting reaching only 53%, and failures stem mainly from incomplete procedures, methodological deviations, and execution fragility.
arXiv AI
Sep 18
ScientistTwo: Pioneering the Human Knowledge Frontier with Autonomous AI
The paper introduces ScientistTwo, a fully autonomous multi‑agent framework that takes a scientific problem, establishes baselines, generates hypotheses, and coordinates specialized agents to conduct an end‑to‑end discovery cycle without human intervention. It rigorously tests and refines its methods through automated experiments, ablation studies, and a closed‑loop peer‑review engine. Benchmarking against top conferences (ICLR, ICML, NeurIPS) shows that ScientistTwo produces expert‑level, publishable papers and codebases that outperform human state‑of‑the‑art models and receive higher review ratings under automated AI review.
arXiv AI
Jun 8
Act As a Real Researcher: A Suite of Benchmarks Evaluating Frontier LLMs and Agentic Harnesses in Research Lifecycle
arXiv:2606. 07462v1 Announce Type: new Abstract: As foundation models advance and agent scaffolding becomes increasingly sophisticated, agents have demonstrated remarkable proficiency in complex, long-horizon coding tasks and even autonomous experiment execution.
arXiv AI
Jun 16
Towards End-to-End Automation of AI Research
arXiv:2606. 15497v1 Announce Type: new Abstract: The automation of science is a long-standing ambition in the field of AI.
arXiv AI
Jun 24
CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmark
arXiv:2409. 11363v2 Announce Type: replace-cross Abstract: AI agents have the potential to aid users on a variety of consequential tasks, including conducting scientific research.
arXiv AI
Jun 12
Benchmarking AI Agents for Addressing Scientific Challenges Across Scales
arXiv:2606. 12736v1 Announce Type: new Abstract: AI agents are increasingly being developed to accelerate scientific discovery, yet their practical capabilities in real research settings remain poorly understood.