Sci‑MMR is a new benchmark for multi‑step evidence‑grounded scientific reasoning in multimodal agents, featuring 235 multi‑hop tasks across four disciplines and an average of nine figure panels per task. It evaluates not just final answer accuracy but also the recovery of structured evidence from scientific claims, citations, visual data, and supporting regions. Experiments on eight state‑of‑the‑art models show a gap of over 20 points between answer accuracy and complete evidence recovery, highlighting significant challenges in evidence acquisition and integration.
By Jiaqiang Li, Yajie Yang, Zhiheng Xi, Jiadong Chen, Enyu Zhou, Senjie Jin, Yang Nan, Jiazheng Zhang, Han Wang, Yanxin Li, Dingwei Zhu, Bicheng Deng, Yuhui Wang, Xiang Zheng, Qi Zhang, Lei Bai, Xingjun Ma, Tao Gui
BioEVAL is a global, multi‑institutional benchmark that evaluates large language and multimodal models on experimental reasoning tasks in bioengineering. It comprises 608 items across 11 bioengineering subfields, including 359 vetted multiple‑choice questions, 218 literature synthesis tasks, and 10 multimodal image‑interpretation problems. The benchmark was used to assess models such as ChatGPT, Gemini, and Grok, revealing accuracies up to 90% on MCQs, a 0.72 similarity score on literature synthesis, and 80% accuracy on multimodal tasks, with notable variation across subfields.
By Shun Ye, Vinny Chandran Suja, Chenlong Li, Chongming Jiang, Reza Zamani, Xiang Li, Christopher Bain, Yuqi Zhou, Walker Peterson, Huidong Wang, Chenglang Hu, Jongchan Park, Xiao Cheng, Benjamin Swedlund, Sandra Murillo, Anjali Sivanandan, Shiyu Sun, Liang Lanfeng, Mohammad Tariqul Islam, Baju C. Joy, Ishaq N. Khan, Sreedhar S. Kumar, Gabriel Mercado-V\'asquez, James V. Vizzard, Jonathan M. Matthews, Helen Huang, Xiaolu Guo, Ethan Nicklow, Guorui Chen, Ryan A. Neff, Surjendu Maity, Hyeonjin Park, Han-ho Joo, Katherine Dong, Yuyan Cai, Weihang Huang, Yichen Zou, Rui Yan, Raphael Figueroa, Artem Goncharov, Bella Rose Schremmer, Lian Elsa Linton, Keisuke Goda, Liang Gao, Ke Cheng, Leonardo Morsut, Jennifer L. Wilson, Jianping Fu, Lim Chwee Teck, Deblina Sarkar, Andreas Hierlemann, Sava\c{s} Tay, Alexander Hoffmann, Donald Richieri Griffin, Jun Chen, Shana O. Kelley, Shyni Varghese, Jinwoo Cheon, Wilbur A. Lam, James J. Moon, Wilson W. Wong, Samir Mitragotri, Dino Di Carlo
arXiv:2509.21193v2 Announce Type: replace
Abstract: Large language models (LLMs) have recently shown strong progress on scientific reasoning, yet two major bottlenecks remain. First, explicit retriev...
By Xiangru Tang, Wanghan Xu, Yujie Wang, Zijie Guo, Daniel Shao, Jiapeng Chen, Cixuan Zhang, Ziyi Wang, Lixin Zhang, Guancheng Wan, Wenlong Zhang, Lei Bai, Zhenfei Yin, Philip Torr, Hanrui Wang, Di Jin
arXiv:2603.00889v2 Announce Type: replace-cross
Abstract: Large Language Models (LLMs) have exhibited remarkable reasoning capabilities, largely enabled by supervised fine-tuning (SFT)- and reinforce...
By Xinyu Zhu, Yihao Feng, Yanchao Sun, Xianzhi Du, Pingzhi Li, Olli Saarikivi, Yun Zhu, Yu Meng
The paper introduces OmicsBench, a new reasoning benchmark for multi‑omics sequences that includes 1,160 expert‑validated questions across DNA regulation, RNA processing, and protein function tasks, requiring traceable evidence chains. Evaluation of 17 large language models shows that scientific LLMs, while more accurate in classification, often lack valid evidence, suggesting shortcut learning. To address this, the authors propose tool‑augmented on‑policy distillation (TA‑OPD), a post‑training method that improves both evidence grounding and predictive performance across five Qwen3.5 models of varying sizes.
By Jie Ying, Zhefan Wang, Zihong Chen, Zhengqing Li, Jinzhe Li, Gang Li, Jian Liu, Fang Hu, Tao Luo, Zhonghang Yuan, Wanli Ouyang, Stan Z. Li, Fan Yang, Nanqing Dong
arXiv:2604. 13201v2 Announce Type: replace-cross Abstract: Large language models are emerging as scientific assistants, but evaluating their ability to reason from empirical data remains challenging.
By Oliver Bentham, Vivek Srikumar
arXiv:2606. 10381v1 Announce Type: cross Abstract: Muon collider research spans accelerator physics, detector instrumentation, and high-energy phenomenology, with relevant evidence scattered across a rapidly expanding and heterogeneous body of scientific literature.
By Ruobing Jiang, Dawei Fu, Cheng Jiang, Tianyi Yang, Zijian Wang, Youpeng Wu, Yong Ban, Yajun Mao, Qiang Li
arXiv:2607. 01131v1 Announce Type: cross Abstract: Autonomous scientific discovery systems offer the potential to accelerate research by automating the process of hypothesis generation and validation.
By Bingchen Zhao, Sara Beery, Oisin Mac Aodha
The paper introduces the Large Knowledge Model (LKM), a scientific knowledge infrastructure that converts research literature into shared, computationally accessible reasoning graphs. LKM aligns questions, claims, and reasoning chains across papers, creating a Scientific Reasoning Landscape with Question, Workflow, and Evidence views. The system enhances scientific search, evidence‑grounded QA, and research planning, achieving notable accuracy gains on ChemBench, PubMedQA, and SciBench.
By Yuan Huang, Sihan Hu, Hongyu Gu, Chao Ma, Jiaxing Zhang, Zhiyong Zou, Caiyu Fan, Yan Xiao, Mingjun Xu, Chenyu Xie, Mingzhen Ju, Zhehao Ma, Qi Zhang, Baozong Wang, Yu Li, Zhiyuan Yao, Ruoxue Liao, Xinyu Li, Linfeng Zhang, Kun Chen, Weinan E
SciRerankBench is a benchmark designed to evaluate rerankers in retrieval‑augmented large language models (RAG‑LLMs) for scientific literature question answering. It covers five scientific subjects and introduces three types of question‑context‑answer pairs—Noisy Contexts, Semantically Similar but Logically Irrelevant Contexts, and Counterfactual Contexts—to test noise resilience, relevance disambiguation, and factual consistency. The study evaluates 13 popular rerankers across five LLM families, offering detailed insights into their strengths and limitations.
By Haotian Chen, Qingqing Long, Meng Xiao, Xiao Luo, Wei Ju, Chengrui Wang, Yuanchun Zhou, Xuezhi Wang, Hengshu Zhu
arXiv:2608.30214v1 Announce Type: new
Abstract: Scientific reasoning remains challenging for open-source models, largely due to the lack of high-quality scientific reasoning data. Existing datasets a...
By Yu Li, Wei Li, Xin Gao, Mengyuan Sun, Xiaoyang Wang, Qizhi Pei, Lijun Wu
OmniVCBench is a figure‑centric, source‑traceable benchmark designed to evaluate the interpretation component of Artificial Intelligence Virtual Cells (AIVCs). It comprises 6,077 curated question–answer pairs drawn from scientific figures and experimental contexts, organized into three scientific reasoning tasks that mirror the AIVC Predict–Explain–Discover agenda. The benchmark also introduces AIVC‑Judge, a task‑conditioned MLLM‑as‑a‑judge framework with reference‑aware rubrics, and a Model‑Derived Hard‑Negative Mining strategy to generate multiple‑choice distractors for efficient evaluation.
By Manyu Li, Xunkai Li, Yongfu Xiong, Yi Liu, Rong-Hua Li, Guoren Wang