The paper introduces ChemCost, a benchmark of 1,427 reactions that tests large language models’ ability to estimate chemical procurement costs by grounding identities, retrieving supplier quotes, selecting purchasable packs, normalizing quantities, and computing costs. It includes a frozen pricing snapshot of 2,261 chemicals and 230,775 supplier quotes, and provides scalar scoring plus stage‑level diagnostics for grounding, retrieval, procurement, and arithmetic errors. Experiments show that even advanced LLM agents achieve only about 50.6% accuracy within 25% relative error on clean inputs, and performance drops sharply with realistic noise, highlighting brittleness in parsing, evidence integration, pack selection, and tool use.
arXiv:2606. 03660v1 Announce Type: new Abstract: Large language models are increasingly used as chemistry assistants, yet most chemistry benchmarks still score only final answers.
RxnOptBench is a new benchmark that tests large language models on the task of selecting optimal reaction conditions—such as catalyst, ligand, solvent, temperature, and atmosphere—by reading real condition‑screening tables from 2025 organic‑methodology papers. The benchmark uses a continuous relative score that combines yield with stereochemical metrics (ee, dr, rr) and includes a design that separates in‑context literature use from mere memorization. Evaluation across nine frontier LLMs and three chemistry‑specialized LLMs shows that even the best models still have significant room for improvement, with chemistry‑specialized models performing at random levels on multi‑axis selection while open‑weight models approach proprietary models.
The paper introduces ChemDiffAgent, a masked diffusion language agent designed for tool‑integrated chemical reasoning. Unlike traditional autoregressive agents, it treats reasoning as iterative denoising over interleaved reasoning and tool‑use trajectories, allowing joint decision making under full‑trajectory context. A two‑stage post‑training framework further refines tool use and reasoning, and the authors benchmark the model on six chemistry tasks, reporting a ~20% performance gain over autoregressive baselines and competitive results against larger models.
MolDesignBench is a new benchmark for evaluating large language model (LLM)-based agents in scenario‑grounded molecular design. It contains 2,000 generation and optimization tasks that blend implicit narrative requirements with explicit property and functional‑group constraints, including infeasible cases, and require the use of 17 specialized chemistry tools. Experiments with leading LLMs show low success rates (best ~43%) and highlight failures in implicit‑constraint reasoning, infeasibility detection, and tool usage, underscoring the benchmark’s role in identifying key bottlenecks for future research.
The paper introduces MAGI, an open modular agent that orchestrates molecular‑design tools by setting objectives, launching optimization, interpreting structure–activity relationships, and adjusting strategy. In nine retrospective lead‑optimization campaigns from three pharmaceutical companies, MAGI produced valid molecules via either direct LLM generation or delegation to REINVENT 4; LLM proposals stayed closer to local chemistry and achieved comparable or higher activity with fewer operations, while REINVENT explored broader chemical space. The success of a campaign depended on the accuracy of predictive models rather than the generation route, and a blinded evaluation showed chemists could not distinguish MAGI’s outputs from expert‑generated compounds, though the SAR reasoning was deemed plausible yet incomplete. The study positions MAGI as a coordination layer that can be plugged into existing workflows, but notes that the current ceiling for real projects is set by scorer applicability rather than tool orchestration.
The paper introduces an LLM-as-a-Judge framework for evaluating the outputs of an agentic drug discovery assistant, ChatInvent, deployed at AstraZeneca. It defines four quality dimensions—Completeness, Relevancy, Structural Clarity, and Scope Adherence—alongside deterministic Tool Call Correctness checks, and validates the judge against five expert annotators. After optimizing the best-performing judge with few-shot demonstrations, alignment with human majority votes improves from 0.80 to 0.86, and the framework reveals that informal question phrasing does not degrade output quality.
oMeBench is a large-scale, expert-curated benchmark designed to evaluate large language models (LLMs) on organic mechanism reasoning. It contains over 10,000 annotated mechanistic steps, including reaction type labels, intermediate structures, and difficulty ratings, and introduces the oMeS scoring framework to assess logical consistency and chemical structural similarity. Evaluation shows that while current LLMs display promising chemical intuition, they often fail to produce correct and consistent multi-step reasoning, though prompting and fine-tuning can bring smaller models up to the level of closed‑source frontier models.
arXiv:2610.17183v1 Announce Type: new Abstract: Current chemistry benchmarks primarily evaluate outcome prediction or reaction pathway generation, but rarely test whether a model can explain why an o...
arXiv:2508.05427v2 Announce Type: replace Abstract: Large language models (LLMs) are beginning to reshape how organic-synthesis workflows are represented, queried, planned, and connected to experimen...
arXiv:2608. 02595v1 Announce Type: new Abstract: Language models are playing an increasingly important role in laboratory science, performing tasks such as experiment planning, execution, and post-hoc analysis.
arXiv:2602.00663v3 Announce Type: replace Abstract: Optimizing molecules to achieve desired properties is a central bottleneck across the chemical sciences, particularly in the pharmaceutical industr...
La Agente ’Optima is an agentic framework that builds and manages Bayesian optimization campaigns for self‑driving laboratories, separating large language model reasoning from campaign execution. It maintains a persistent optimization state, allowing consistent repetitive loops and auditable decisions, and only returns control to the agent when interpretation or revision is needed. In tests on digital discovery tasks and physical platforms, it corrected measurement failures, improved yields, and recommended formulation changes, outperforming human‑directed campaigns in cost and material usage.