The Flow has not summarised this story yet — read it at arXiv Machine Learning.
arXiv:2606. 03660v1 Announce Type: new Abstract: Large language models are increasingly used as chemistry assistants, yet most chemistry benchmarks still score only final answers.
arXiv:2610.17183v1 Announce Type: new Abstract: Current chemistry benchmarks primarily evaluate outcome prediction or reaction pathway generation, but rarely test whether a model can explain why an o...
ChemVTS-Bench is a domain-authentic benchmark that evaluates Visual‑Textual‑Symbolic reasoning in multimodal large language models for chemistry. It presents diverse chemical problems—organic molecules, inorganic materials, and 3D crystal structures—in three input modes: visual-only, visual‑text hybrid, and SMILES-based symbolic. The benchmark includes an automated agent workflow for inference, answer verification, and failure diagnosis, and shows that visual-only inputs and structural chemistry remain challenging for current models.
The paper introduces ChemDiffAgent, a masked diffusion language agent designed for tool‑integrated chemical reasoning. Unlike traditional autoregressive agents, it treats reasoning as iterative denoising over interleaved reasoning and tool‑use trajectories, allowing joint decision making under full‑trajectory context. A two‑stage post‑training framework further refines tool use and reasoning, and the authors benchmark the model on six chemistry tasks, reporting a ~20% performance gain over autoregressive baselines and competitive results against larger models.
arXiv:2510. 12171v2 Announce Type: replace Abstract: Large Language Models have shown strong scientific reasoning ability, but their performance on materials science problems remains less studied.
BioEVAL is a global, multi‑institutional benchmark that evaluates large language and multimodal models on experimental reasoning tasks in bioengineering. It comprises 608 items across 11 bioengineering subfields, including 359 vetted multiple‑choice questions, 218 literature synthesis tasks, and 10 multimodal image‑interpretation problems. The benchmark was used to assess models such as ChatGPT, Gemini, and Grok, revealing accuracies up to 90% on MCQs, a 0.72 similarity score on literature synthesis, and 80% accuracy on multimodal tasks, with notable variation across subfields.