arXiv AI August 26, 2026 By Qiang Xu, Shengyuan Bai, Yu Wang, He Cao, Leqing Chen, Yuanyuan Liu, Bin Feng, Zijing Liu, Yu Li

ReactBench: A Benchmark for Topological Reasoning in MLLMs on Chemical Reaction Diagrams

Read the original on arXiv AI →

ReactBench is a benchmark designed to evaluate the structural reasoning abilities of multimodal large language models (MLLMs) using chemical reaction diagrams. The dataset contains 1,618 expert‑annotated question‑answer pairs that test reasoning across four hierarchical task dimensions, from simple endpoint counting to complex topological analysis. Evaluation of 24 MLLMs shows a performance gap of more than 30% between anchor‑based tasks and holistic structural reasoning tasks, indicating that current models struggle with reasoning over branching, converging, and cyclic structures.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

1d ago

oMeBench: Towards Robust Benchmarking of LLMs in Organic Mechanism Elucidation and Reasoning

arXiv:2510.07731v4 Announce Type: replace Abstract: Organic reaction mechanisms describe the step-wise elementary processes by which reactants transform into intermediates and products, and are funda...

By Ruiling Xu, Yifan Zhang

llms fine-tuning benchmarks

arXiv AI

Jul 17

TopoAgent: A Self-Evolving Topological Agent for Multimodal Scientific Reasoning

arXiv:2607. 14658v1 Announce Type: new Abstract: While Multimodal Large Language Models (MLLMs) excel in general tasks, rigorous scientific reasoning remains challenging due to the limitations of monolithic, linear planning.

By Mingze Xu, Yinghui Li, Jiayi Kuang, Zhanhui Kang, Di Yin, Ying Shen, Xing Sun, Yuxing Han

llms agents multimodal benchmarks safety

arXiv Machine Learning

Jul 15

Learning Mechanistic Reasoning for Chemical Reactions with Large Language Models

arXiv:2607. 12771v1 Announce Type: new Abstract: Reaction mechanisms consist of the step-by-step sequences of elementary reactions that explain chemical transformations.

By Xingyu Dang, Haocheng Tang, Junmei Wang, Yanjun Li

llms fine-tuning benchmarks safety

arXiv Machine Learning

Jul 2

VisReason: A Large-Scale Dataset for Visual Chain-of-Thought Reasoning

arXiv:2511. 17731v2 Announce Type: replace-cross Abstract: Chain-of-Thought (CoT) prompting has proven remarkably effective for eliciting complex reasoning in large language models (LLMs).

By Lingxiao Li, Yifan Wang, Xinyan Gao, Chen Tang, Xiangyu Yue, Chenyu You

llms fine-tuning multimodal benchmarks safety

arXiv AI

Jul 7

Back to Basics: Improving Molecular Understanding in LLMs via SMILES-Graph Translation

arXiv:2607. 03007v1 Announce Type: cross Abstract: Recent advances in molecular large language models have led to strong performance on molecular understanding and generation tasks, yet these gains often come without reliable structural grounding.

By Wenda Wang, Jinjia Feng, Zhewei Wei

llms benchmarks

arXiv Computation and Language

Sep 10

From Symbolic Perception to Logical Deduction: A Framework for Guiding Language Models in Geometric Reasoning

The paper introduces a framework that combines a Geometric Vision Parser and a Symbolic Solver to enable a Large Language Model to solve complex plane geometry problems. By translating diagrams into symbolic representations and performing formal deductions, the approach reduces hallucinations and produces interpretable, human-like solutions. Experiments on a new benchmark from 2025 Chinese Zhongkao exams show performance comparable to Gemini 2.5 Pro.

By Weichen Dai, Rafael Medeiros Cabral, Ziyi Shou, Yan Cao, Xin Shen, Dongcai Lu, Yi Zhou

llms multimodal benchmarks safety

One email a morning, machine-written

Related stories

oMeBench: Towards Robust Benchmarking of LLMs in Organic Mechanism Elucidation and Reasoning

TopoAgent: A Self-Evolving Topological Agent for Multimodal Scientific Reasoning

Learning Mechanistic Reasoning for Chemical Reactions with Large Language Models

VisReason: A Large-Scale Dataset for Visual Chain-of-Thought Reasoning

Back to Basics: Improving Molecular Understanding in LLMs via SMILES-Graph Translation

From Symbolic Perception to Logical Deduction: A Framework for Guiding Language Models in Geometric Reasoning