arXiv AI September 12, 2026 By Jiaqiang Li, Yajie Yang, Zhiheng Xi, Jiadong Chen, Enyu Zhou, Senjie Jin, Yang Nan, Jiazheng Zhang, Han Wang, Yanxin Li, Dingwei Zhu, Bicheng Deng, Yuhui Wang, Xiang Zheng, Qi Zhang, Lei Bai, Xingjun Ma, Tao Gui

Sci-MMR: Benchmarking Multi-Step Evidence-Grounded Scientific Reasoning in Multimodal Agents

Read the original on arXiv AI →

Sci‑MMR is a new benchmark for multi‑step evidence‑grounded scientific reasoning in multimodal agents, featuring 235 multi‑hop tasks across four disciplines and an average of nine figure panels per task. It evaluates not just final answer accuracy but also the recovery of structured evidence from scientific claims, citations, visual data, and supporting regions. Experiments on eight state‑of‑the‑art models show a gap of over 20 points between answer accuracy and complete evidence recovery, highlighting significant challenges in evidence acquisition and integration.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jul 17

Building Agent Harnesses for Scientific Curation from Multimodal Sources

arXiv:2606. 21005v2 Announce Type: replace Abstract: Scientific discovery workflows often depend on structured curation from the literature.

By Sheng Zhang, Qin Liu, Renqian Luo, Shufang Xie, Reuben Tan, Sean Hayes, Gregory Bryman, Wendong Ge, Ruilian Zhang, Oluwaseun Egbelowo, Kelly Yee, Hoifung Poon

agents multimodal

arXiv AI

2d ago

Mr.LHDR: A Benchmark for Multimodal Real-World Long-Horizon Deep Research Agents

Mr.LHDR is a new benchmark designed to evaluate deep research agents on long‑horizon, multimodal tasks. It presents questions built from hidden Node‑Relation graphs that require an average of 12.1 intermediate conclusions and a mean dependency depth of 10.4 before arriving at a single verifiable answer. The benchmark tests both final answers and the correctness of intermediate conclusions, using metrics such as Overall Accuracy, Strict Accuracy, Checklist Score, and Dependency‑Aware Checklist Score.

By Minghao Guo, Meng Cao, Sui Zhao, Siyu Ning, Xin Wang, Haoze Zhao, Jiaxuan Yang, Haihong Hao, Mingfei Han, Shunlin Rong, Haijun Wu, Xiaodan Liang, Xiaojun Chang

agents multimodal benchmarks

arXiv AI

Jun 12

SciR: A Controllable Benchmark for Scientific Reasoning in LLMs

arXiv:2606. 13020v1 Announce Type: new Abstract: Three paradigmatic forms of inference recur across scientific reasoning: deduction, induction, and causal abduction.

By Pierre Beckmann, Marco Valentino, Andre Freitas

llms benchmarks

arXiv AI

Jul 20

ToolSciVer: Multimodal Scientific Claim Verification with Visual Tool Augmented Reinforcement Learning

arXiv:2607. 16131v1 Announce Type: cross Abstract: Multimodal Scientific Claim Verification (MSCV) requires models to verify scientific claims using visually grounded evidence from papers, including figures, tables, charts, and textual context.

By Binglin Zhou, Peng Shi, Ryo Kamoi, Nan Zhang, Rui Zhang

llms agents reinforcement-learning multimodal

arXiv AI

Aug 11

MMArch: Benchmarking Multimodal Reasoning Grounded in Architectural Evidence

arXiv:2608. 09281v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) perform strongly on engineering imagery, yet existing benchmarks mostly test drawing recognition, information extraction, or compliance checking, leaving open whether models can combine distributed visual evidence with engineering principles to reach a conclusion.

By Chenxu Du, Kang An, Tengyue Wang, Zhongyu Yang, Xinqi Yang, Yuanchi Zhu, Hebao Zhu, Ziliang Wang, Faqiang Qian, Yunli Yang, Qibing Ren

llms multimodal benchmarks safety

arXiv AI

Jul 2

Autonomous Scientific Discovery via Iterative Meta-Reflection

arXiv:2607. 01131v1 Announce Type: cross Abstract: Autonomous scientific discovery systems offer the potential to accelerate research by automating the process of hypothesis generation and validation.

By Bingchen Zhao, Sara Beery, Oisin Mac Aodha

llms agents multimodal benchmarks

One email a morning, machine-written

Related stories

Building Agent Harnesses for Scientific Curation from Multimodal Sources

Mr.LHDR: A Benchmark for Multimodal Real-World Long-Horizon Deep Research Agents

SciR: A Controllable Benchmark for Scientific Reasoning in LLMs

ToolSciVer: Multimodal Scientific Claim Verification with Visual Tool Augmented Reinforcement Learning

MMArch: Benchmarking Multimodal Reasoning Grounded in Architectural Evidence

Autonomous Scientific Discovery via Iterative Meta-Reflection