arXiv AI
Aug 12
Automating and Scaling Behavioral Scientific Research on AI Agents
arXiv:2608. 10030v1 Announce Type: new Abstract: As AI agents are increasingly deployed in complex environments, understanding their behaviors becomes critical.
Automating and Scaling Behavioral Scientific Research on AI Agents
As AI agents are increasingly deployed in complex environments, understanding their behaviors becomes critical. Yet behavioral scientific research on AI agents remains manual and labor-intensive.
Related stories
arXiv AI
Jul 21
From Overload to Insights: How AI Agents Can Support Scientists in Analyzing Complex Data
arXiv:2607. 16845v1 Announce Type: new Abstract: Scientists at European XFEL conduct experiments that generate very large and complex datasets.
arXiv AI
Jun 12
Benchmarking AI Agents for Addressing Scientific Challenges Across Scales
arXiv:2606. 12736v1 Announce Type: new Abstract: AI agents are increasingly being developed to accelerate scientific discovery, yet their practical capabilities in real research settings remain poorly understood.
arXiv AI
Jun 11
Preregistration for Experiments with AI Agents
arXiv:2606. 11217v1 Announce Type: cross Abstract: The proliferation of large language models (LLMs) and autonomous AI agents has given rise to a rapidly growing methodological paradigm: "in silico" behavioral experiments.
arXiv AI
Jun 8
Act As a Real Researcher: A Suite of Benchmarks Evaluating Frontier LLMs and Agentic Harnesses in Research Lifecycle
arXiv:2606. 07462v1 Announce Type: new Abstract: As foundation models advance and agent scaffolding becomes increasingly sophisticated, agents have demonstrated remarkable proficiency in complex, long-horizon coding tasks and even autonomous experiment execution.
arXiv AI
Jun 16
Towards End-to-End Automation of AI Research
arXiv:2606. 15497v1 Announce Type: new Abstract: The automation of science is a long-standing ambition in the field of AI.
arXiv AI
Jun 4
SciDER: Scientific Data-centric End-to-end Researcher
arXiv:2603. 01421v3 Announce Type: replace Abstract: While large language models accelerate scientific discovery, existing agents face severe limitations in adaptability, domain generalization, and multimodal scalability, often struggling to autonomously process raw, domain-specific experimental data.
OpenAI Blog
Apr 2, 2025
PaperBench: Evaluating AI’s Ability to Replicate AI Research
We introduce PaperBench, a benchmark evaluating the ability of AI agents to replicate state-of-the-art AI research.
arXiv AI
Jun 26
auto-psych: Automating the science of mind using agent-driven theory discovery and experimentation
arXiv:2606. 26460v1 Announce Type: new Abstract: AI-based scientific automation is increasingly possible by using agents to generate hypotheses, design experiments, and analyze data.
arXiv AI
Jul 1
ReplicatorBench: Benchmarking LLM Agents for Replicability in Social and Behavioral Sciences
arXiv:2602. 11354v3 Announce Type: replace Abstract: The literature has witnessed an emerging interest in AI agents for automated assessment of scientific papers.
arXiv AI
Jun 9
ResearchClawBench: A Benchmark for End-to-End Autonomous Scientific Research
arXiv:2606. 07591v1 Announce Type: cross Abstract: AI coding agents are increasingly used for scientific work, but their end-to-end autonomous research capability remains difficult to verify.
arXiv AI
Jun 4
Can Generalist Agents Automate Data Curation?
arXiv:2606. 04261v1 Announce Type: new Abstract: Curating training data is among the most consequential yet labor-intensive parts of modern AI development: practitioners iteratively propose, implement, evaluate, and revise data policies against noisy benchmark feedback.