BenchBench-Protocol is a new benchmark for large language models that evaluates their ability to modify real-world wet‑lab protocols. It consists of 149 protocol‑modification tasks derived from actual changes scientists made to published protocols across 96 source protocols in nine wet‑lab biology domains. The benchmark includes weighted rubric elements for correct responses and has been reviewed by domain experts to ensure high quality.
By Aditya Sivakumar, Ashu Singhal, Nicholas Larus-Stone, Nithin Parsan
The paper introduces LibStructBench, a document‑grounded benchmark designed to assess whether AI agents can accurately reconstruct sequencing‑library structures and molecular workflows from protocols. Experiments reveal that current state‑of‑the‑art agents often generate roughly correct reconstructions but struggle with exact molecular‑state representation and workflow connectivity. The study also finds that changing the underlying language model yields larger performance differences than changing agent harnesses, and that additional knowledge only modestly improves oligo‑family recovery without affecting overall workflow accuracy.
By Chi-Lam Poon, Michael Goneos, Zezheng Song, Xi Chen
arXiv:2608.31076v1 Announce Type: cross
Abstract: Autonomous scientific research agents are increasingly applied to end-to-end scientific workflows, including literature review, data analysis, experi...
By Xuehai Wang, Haowei Qin, Tongxin Liu, Junkai Li, Buqiang Xu, Jintian Zhang, Yijun Chen, Zirui Xue, Shumin Deng
FrontierChallenge is a cross‑domain benchmark that releases 300 end‑to‑end scientific workflows, of which 97 are evaluated in this study. The benchmark covers diverse fields such as quantum chemistry, molecular dynamics, materials characterization, analytical chemistry, life science, and electrochemistry/environment, and requires agents to produce a bundle of fixed scientific deliverables. Twelve frontier models were tested, and the best configurations completed only 20 of the 97 tasks, achieving a 20.6% pass rate; high partial scores and confident completion claims often did not translate into full delivery, especially in analytical chemistry and electrochemistry/environment.
By Liangcai Su, Zhaopeng Feng, Zhuo Chen, Zhen Zhang, Xiang Lin, Ruilin Li, Handuo Zhang, Ning Wang, Kailong Wen, Yueqi Guo, Feng Xing, Yiling Guo, Chenxiong Qian, Simon Shaolei Du, Lidong Bing, Xinyu Wang
PAPERBENCHX is a new benchmark that tests whether AI can reproduce scientific experiments across 12 disciplines, including electromagnetics, photonics, chemistry, materials science, biology, and robotics. It consists of 93 expert-constructed tasks, each providing a source paper, a public specification, and a domain-native scientific environment. The benchmark evaluates AI agents by having them reconstruct, execute, and diagnose key experiments, then replay submissions from scratch to score regenerated evidence against expert-validated criteria, revealing that current AI configurations fully reproduce only 14% of tasks.
By Ruoyu Wu, Zengji Tu, Aolong Sun, Tianyi Ma, Junren Li, Xinbo Xu, Li Fu, Ziyan Zhang, Pinrui Huang, Xiaochen Wang, Chengda Wu, Yutao Zhou, Jianjun Wu, Lin Chang, Baobao Chang, Liang Chen, Kuan Li
The Bioinfoysis Technical Report introduces a multi‑agent harness designed to improve long‑horizon bioinformatics tasks by maintaining persistent, artifact‑grounded analysis runs. It combines global planning with step‑wise, evidence‑driven replanning, ensuring intermediate results are tied to responsible agents and preventing stale evidence reuse. The system was evaluated on BixBench and LAB‑Bench 2, achieving state‑of‑the‑art accuracy and demonstrating that reliable bioinformatics automation relies on robust planning, execution, memory, and evidence flow.
By Qingyang Shao, Xin Zhang, Zhouyang Yuan, Xianying Chen, Yujia Xiang, Zihao Yang, Tong Ye, Yangqi Zhang, Jiakang Xu, Xiaoqing Yan, Xuan Luo, Keyi Li, Enci Fan, Kai Kang, Zhuohan Liu, Xingyu Jin, Chunran Teng, Tao Li, Xinyu Lv, Minghui Wang, Wenfeng Li, Yidan Gao, Siyu Liu, Mingrui Luo, Zhu Liang, Guanren Qiao, Zhiping Xu