arXiv AI October 2, 2026 By Xin Wang, Zichuan Ying, Xinna Lin, Junqi Zhang, Hanyi Xiong, Tianyu Gao, Hairong Zhang, Qixiang Hua, Botian Shi, Zhenhailong Wang, Kaicheng Yu

R-GroundBench: A Diagnostic Benchmark for R-Group Groundingin Markush Molecular Editing

Read the original on arXiv AI →

R-GroundBench is a new diagnostic benchmark for evaluating AI models on R‑group grounding in Markush molecular editing, derived from real pharmaceutical patents. It includes a Multiple‑Choice VQA track with varying difficulty and modality splits, as well as an open‑ended Generation track. Experiments show a large performance gap: models score over 90% on easy VQA but drop to 56–66% on hard VQA, and generation exact match stays below 20% (and under 8% with visual input).

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

benchmarks

arXiv AI

Jun 3

From Answers to States: Verifiable Process-Level Evaluation of Chemical Reasoning in Large Language Models

arXiv:2606. 03660v1 Announce Type: new Abstract: Large language models are increasingly used as chemistry assistants, yet most chemistry benchmarks still score only final answers.

By Hongyu Guo, Hao Li, He Cao, Gongbo Zhang, Li Yuan

llms benchmarks safety

arXiv AI

Sep 24

MolDesignBench: Evaluating LLM-based Agent for Scenario-grounded Molecular Design

MolDesignBench is a new benchmark for evaluating large language model (LLM)-based agents in scenario‑grounded molecular design. It contains 2,000 generation and optimization tasks that blend implicit narrative requirements with explicit property and functional‑group constraints, including infeasible cases, and require the use of 17 specialized chemistry tools. Experiments with leading LLMs show low success rates (best ~43%) and highlight failures in implicit‑constraint reasoning, infeasibility detection, and tool usage, underscoring the benchmark’s role in identifying key bottlenecks for future research.

By Yongjun Jeong, Hanbum Ko, Ye Rin Kim, Chanhui Lee, Rodrigo Hormazabal, Jaewan Lee, Sehui Han, Sungbin Lim, Sungwoong Kim

llms agents benchmarks

arXiv AI

Aug 19

Domain-Adapted Molecular Language Models for Efficient Search of Make-on-Demand Libraries

The study evaluates four pretrained molecular language models on six virtual libraries covering drug discovery, organic materials, and catalysis. It finds that native embeddings vary widely in performance, while molecular fingerprints remain consistently strong. Fine‑tuning the models on library‑specific data markedly improves sample efficiency, with several adapted encoders outperforming others across all tasks.

By Henrik Wille, Luis-Finley Sch\"utz, Felix Strieth-Kalthoff

llms rag robotics fine-tuning benchmarks

arXiv AI

Jun 3

Rethinking Molecular Text Representations for LLMs: An Empirical Study

arXiv:2606. 03057v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used for molecular tasks, but it remains unclear which molecular representation to use.

By Arun Raja, Garrett M. Morris, Kian Ming A. Chai

llms nlp benchmarks

arXiv AI

Jun 12

SupraBench: A Benchmark for Supramolecular Chemistry

arXiv:2606. 13477v1 Announce Type: cross Abstract: Supramolecular chemistry, which includes the study of non-covalent host-guest assemblies, has advanced various applications.

By Tianyi Ma, Yijun Ma, Zehong Wang, Weixiang Sun, Ziming Li, Connor R. Schmidt, Chuxu Zhang, Matthew J. Webber, Yanfang Ye

llms fine-tuning benchmarks

arXiv AI

Aug 24

SEISMO: Explanation-Aware, Trajectory-Conditioned LLM Agents for Sample-Efficient Molecular Optimisation

arXiv:2602.00663v3 Announce Type: replace Abstract: Optimizing molecules to achieve desired properties is a central bottleneck across the chemical sciences, particularly in the pharmaceutical industr...

By Fabian P. Kr\"uger, Andrea Hunklinger, Adrian Wolny, Tim J. Adler, Igor Tetko, Santiago David Villalba

llms agents

One email a morning, machine-written

Related stories

From Answers to States: Verifiable Process-Level Evaluation of Chemical Reasoning in Large Language Models

MolDesignBench: Evaluating LLM-based Agent for Scenario-grounded Molecular Design

Domain-Adapted Molecular Language Models for Efficient Search of Make-on-Demand Libraries

Rethinking Molecular Text Representations for LLMs: An Empirical Study

SupraBench: A Benchmark for Supramolecular Chemistry

SEISMO: Explanation-Aware, Trajectory-Conditioned LLM Agents for Sample-Efficient Molecular Optimisation