arXiv AI
ChemExplain: A Multimodal Benchmark for Explanatory Chemical Reasoning
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 23
ChemVTS-Bench: Evaluating Visual-Textual-Symbolic Reasoning of Multimodal Large Language Models in Chemistry
ChemVTS-Bench is a domain-authentic benchmark that evaluates Visual‑Textual‑Symbolic reasoning in multimodal large language models for chemistry. It presents diverse chemical problems—organic molecules, inorganic materials, and 3D crystal structures—in three input modes: visual-only, visual‑text hybrid, and SMILES-based symbolic. The benchmark includes an automated agent workflow for inference, answer verification, and failure diagnosis, and shows that visual-only inputs and structural chemistry remain challenging for current models.
arXiv AI
Jun 3
From Answers to States: Verifiable Process-Level Evaluation of Chemical Reasoning in Large Language Models
arXiv:2606. 03660v1 Announce Type: new Abstract: Large language models are increasingly used as chemistry assistants, yet most chemistry benchmarks still score only final answers.
arXiv AI
Sep 18
oMeBench: Towards Robust Benchmarking of LLMs in Organic Mechanism Elucidation and Reasoning
oMeBench is a large-scale, expert-curated benchmark designed to evaluate large language models (LLMs) on organic mechanism reasoning. It contains over 10,000 annotated mechanistic steps, including reaction type labels, intermediate structures, and difficulty ratings, and introduces the oMeS scoring framework to assess logical consistency and chemical structural similarity. Evaluation shows that while current LLMs display promising chemical intuition, they often fail to produce correct and consistent multi-step reasoning, though prompting and fine-tuning can bring smaller models up to the level of closed‑source frontier models.
arXiv AI
Oct 12
Masked Diffusion Language Agents for Tool-Integrated Chemical Reasoning
The paper introduces ChemDiffAgent, a masked diffusion language agent designed for tool‑integrated chemical reasoning. Unlike traditional autoregressive agents, it treats reasoning as iterative denoising over interleaved reasoning and tool‑use trajectories, allowing joint decision making under full‑trajectory context. A two‑stage post‑training framework further refines tool use and reasoning, and the authors benchmark the model on six chemistry tasks, reporting a ~20% performance gain over autoregressive baselines and competitive results against larger models.
arXiv Machine Learning
5d ago
ChemArgus: Automated Scientific Diagnostics for Lab-Grade Multimodal Chemistry Reasoning: A Frontier Benchmark
arXiv:2610.20012v1 Announce Type: new Abstract: Chemistry evaluation has moved past multiple-choice probes toward open-ended expert problems, yet the benchmarks that define the current frontier still...
arXiv AI
Jun 16
Retro-Expert: Collaborative Reasoning for Interpretable Retrosynthesis
arXiv:2508. 10967v3 Announce Type: replace-cross Abstract: Retrosynthesis prediction aims to infer the reactant molecules based on a given product molecule, which is a fundamental task in chemical synthesis.