arXiv:2609.22111v1 Announce Type: new
Abstract: Large language model agents are increasingly capable of conducting research autonomously, producing research documents alongside the code and experimen...
By Qiuhong Shen, Benlong Wu, Hanjin Liu, Yuang Qi, Kejiang Chen
arXiv:2605. 17548v2 Announce Type: replace-cross Abstract: Code review has evolved for decades, from informal peer checking to today's pull request (PR) workflows, yet it remains a largely manual and cognitively demanding process.
By H\"useyin \"Ozg\"ur Kamal{\i}, Erdem Tuna, Vahid Haratian, Eray T\"uz\"un
CrossAudit proposes a Git‑native, cross‑vendor audit protocol for autonomous research pipelines, ensuring each work increment is reviewed by an agent from a different vendor against a human‑written rulebook. Audit outcomes, disputes, and rulings are stored as git commits, providing a replayable, versioned supervision history. The authors implemented the protocol with GitHub Actions and Python, deployed it in a computational‑chemistry pipeline, and conducted a seeded‑defect trial that revealed differing interpretations of the same rulebook by two vendors.
By Zhaohe Dong, Yuhao Chen
PaperDoctor is an agent framework that provides evidence‑grounded, actionable feedback for scientific papers before submission. It evaluates writing, layout, references, code, theory, prior work, and experiments through a three‑layer hierarchical system, linking each critique to specific evidence and revision suggestions. The system selectively rebuilds and reruns experiments to uncover reproducibility gaps, and an interactive interface lets authors explore findings tied to their manuscript.
By Kevin Qinghong Lin, Siyuan Hu, Pan Lu, Yu Chen, Yanzhe Chen, Owen Queen, Yupeng Chen, Jialin Yu, Junchi Yu, Zifeng Ding, Yuanfeng Ji, Sheng Liu, Jindong Gu, Linjie Li, Mike Zheng Shou, Philip Torr, James Zou
arXiv:2506. 08134v4 Announce Type: replace Abstract: Peer review, the bedrock of scientific advancement in machine learning (ML), is strained by a crisis of scale.
By Qiyao Wei, Samuel Holt, Jing Yang, Markus Wulfmeier, Mihaela van der Schaar
The paper "Beyond Execution: Auditing Experimental Fidelity in LLM-Driven Scientific Research" argues that large language model agents must faithfully implement reference methods, design experiments that truly test claims, and provide supporting evidence. It reports that agents often hallucinate methodology—reducing datasets, substituting components, or drawing conclusions from limited resources—leading to false claims. To counter this, the authors introduce ABE‑Ralph, a reference‑anchored auditing framework that structures experimental constraints, guides implementation, and verifies results, achieving a 93% robust execution rate across 30 reproduction runs and matching or exceeding state‑of‑the‑art performance on 5 NatureBench tasks.
"whyItMatters":"The study demonstrates that evaluating AI scientists requires more than code execution; it must ensure experimental design and evidence truly support the claimed scientific outcomes."
By Lezhi Yu, Xiaogang Xu, Yuhua Zhou, Shuibing He, Aimin Pan