arXiv AI
Read the Paper, Write the Code: Agentic Reproduction of Social-Science Results
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Computation and Language
Sep 22
Beyond the Text: Verifying That Agent-Written Papers Are Backed by Their Artifacts
arXiv:2609.22111v1 Announce Type: new Abstract: Large language model agents are increasingly capable of conducting research autonomously, producing research documents alongside the code and experimen...
arXiv AI
Sep 3
Can Coding Agents Reproduce Findings in Computational Materials Science?
The paper introduces AutoMat, a benchmark designed to test large language model (LLM) coding agents on their ability to reproduce claims from computational materials science. AutoMat presents three challenges: reconstructing underspecified procedures, navigating specialized toolchains, and assessing whether the evidence supports a claim. Experiments show that current LLM agents achieve low success rates, with the best setting reaching only 53%, and failures stem mainly from incomplete procedures, methodological deviations, and execution fragility.
arXiv AI
Jun 17
ReproRepo: Scaling Reproducibility Audits with GitHub Repository Issues
arXiv:2606. 18237v1 Announce Type: cross Abstract: Reproducing research results from papers and released code is central to scientific progress.
arXiv AI
Jul 1
ReplicatorBench: Benchmarking LLM Agents for Replicability in Social and Behavioral Sciences
arXiv:2602. 11354v3 Announce Type: replace Abstract: The literature has witnessed an emerging interest in AI agents for automated assessment of scientific papers.
arXiv AI
Jul 3
Coding-agents can replicate scientific machine learning papers
arXiv:2607. 02134v1 Announce Type: new Abstract: Scientific machine learning papers typically make computational claims, e.
arXiv AI
Sep 2
EvoSCM: Scientific Belief Revision Through Causal Model Evolution and Experimentation
arXiv:2609.01526v1 Announce Type: new Abstract: Scientific agents must learn not only how to reason, but also what to believe. However, existing LLM agents typically express scientific hypotheses in...