arXiv Machine Learning
Jul 30
Can AI agents conduct open-ended AI research? Early evidence from two case studies
arXiv:2607. 27191v1 Announce Type: cross Abstract: Forecasts of explosive AI progress hinge on AI agents automating AI research.
Letting AI Formulate the Research Question: Autonomous Research Agents
The paper reports on an autonomous research system that allows an AI agent to formulate its own research question, screen for failure modes, and obtain critiques from a separate model before writing a paper. Over 13 development runs, minor changes in the initial instruction document caused significant shifts in the topics explored, ultimately leading to a new technology applied to a societal problem. Five fully autonomous runs produced reproducible papers that were reviewed by external AI reviewers at workshop level, with all data and code released for replication.
Related stories
arXiv AI
Aug 7
Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap
arXiv:2608. 05179v1 Announce Type: cross Abstract: Large language model (LLM) agents are increasingly used across the scientific research lifecycle: ideation, literature search, experiment design and execution, analysis, manuscript drafting, and review.
arXiv Machine Learning
Oct 15
The Agentic Scientist: Orchestrating Coding Agents for Automated Scientific (Co-)Discovery
arXiv:2610.18933v1 Announce Type: cross Abstract: Modern coding agents are fundamentally reshaping the role of human input for computational research: they write code, launch in silico experiments, a...
arXiv AI
Jun 16
Towards End-to-End Automation of AI Research
arXiv:2606. 15497v1 Announce Type: new Abstract: The automation of science is a long-standing ambition in the field of AI.
arXiv Computation and Language
Sep 16
PaperDoctor: Evidence-Grounded and Actionable Feedback for Scientific Papers in Progress
PaperDoctor is an agent framework that provides evidence‑grounded, actionable feedback for scientific papers before submission. It evaluates writing, layout, references, code, theory, prior work, and experiments through a three‑layer hierarchical system, linking each critique to specific evidence and revision suggestions. The system selectively rebuilds and reruns experiments to uncover reproducibility gaps, and an interactive interface lets authors explore findings tied to their manuscript.
arXiv AI
Jun 2
Can AI Review Improve Paper Drafting? An Empirical Study on 20 Computer Architecture Submissions
arXiv:2606. 01013v1 Announce Type: new Abstract: Research is advancing faster than ever with artificial intelligence (AI); and so are the corresponding research papers.
arXiv AI
Jul 21
AI for Auto-Research: Roadmap & User Guide
arXiv:2605. 18661v2 Announce Type: replace Abstract: AI-assisted research is crossing a threshold: fully automated systems can now generate research papers for as little as $15, while long-horizon agents can execute experiments, draft manuscripts, and simulate critique with minimal human input.
arXiv Computation and Language
Sep 22
Checkpoints Are Not Enough: Trust Calibration in CoSLR, a Human-AI System for Systematic Literature Reviews
The paper introduces CoSLR, a Human‑AI collaborative system for systematic literature reviews that incorporates mandatory human checkpoints within a three‑phase pipeline using large language models and Retrieval‑Augmented Generation. In a survey of 63 participants, 42.9 % rated the system’s usability highly, yet 34.9 % indicated they would trust AI‑generated summaries without further human verification after brief interaction. The study highlights that effective human oversight in AI‑assisted literature reviews depends on users’ willingness to engage with the checkpoints, underscoring a calibration issue that interface design must directly address.
arXiv AI
Jun 8
Act As a Real Researcher: A Suite of Benchmarks Evaluating Frontier LLMs and Agentic Harnesses in Research Lifecycle
arXiv:2606. 07462v1 Announce Type: new Abstract: As foundation models advance and agent scaffolding becomes increasingly sophisticated, agents have demonstrated remarkable proficiency in complex, long-horizon coding tasks and even autonomous experiment execution.
arXiv AI
Sep 15
AppliedScientist: Automated Scientific Revision Through Iterative AI Reviewing
arXiv:2609.14738v1 Announce Type: new Abstract: Automated reviewing systems are increasingly evaluated based on the quality of the reviews they produce. Yet a review is only useful if acting on it le...
arXiv Computation and Language
Sep 17
Notes2Skills: From Lab Notebooks to Certainty-Aware Scientific Agent Skills
arXiv:2606.11897v2 Announce Type: replace Abstract: Scientific discovery workflows rely heavily on lab notes, where researchers record observations, interpret uncertain results, and plan follow-up ex...
arXiv AI
Sep 3
Can Coding Agents Reproduce Findings in Computational Materials Science?
The paper introduces AutoMat, a benchmark designed to test large language model (LLM) coding agents on their ability to reproduce claims from computational materials science. AutoMat presents three challenges: reconstructing underspecified procedures, navigating specialized toolchains, and assessing whether the evidence supports a claim. Experiments show that current LLM agents achieve low success rates, with the best setting reaching only 53%, and failures stem mainly from incomplete procedures, methodological deviations, and execution fragility.