BEAKER is a new benchmark for evaluating embodied actionability, knowledge, experimentation, and reasoning in self‑driving chemical laboratories. It contains 1,000 expert‑curated question‑answer pairs across 11 image and video task types, covering affordance grounding, trajectory planning, equipment and state recognition, safety, apparatus setup, operation, step reasoning, and temporal procedure reasoning. The benchmark is manually created and reviewed by chemistry and embodied AI experts, and it tests 28 different multimodal large language models in an offline, single‑turn setting.
arXiv:2607. 05377v1 Announce Type: cross Abstract: While recent Vision-Language-Action (VLA) models show promise toward generalist manipulation policies, they struggle with long-horizon tasks due to their Markovian nature-relying solely on current observations.
arXiv:2606. 10803v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) excel at utilizing digital APIs and increasingly serve as the "brain" of embodied AI, instructing robots to interact with the physical world.
ChemVTS-Bench is a domain-authentic benchmark that evaluates Visual‑Textual‑Symbolic reasoning in multimodal large language models for chemistry. It presents diverse chemical problems—organic molecules, inorganic materials, and 3D crystal structures—in three input modes: visual-only, visual‑text hybrid, and SMILES-based symbolic. The benchmark includes an automated agent workflow for inference, answer verification, and failure diagnosis, and shows that visual-only inputs and structural chemistry remain challenging for current models.
Multimodal Large Language Models (MLLMs) excel at utilizing digital APIs and increasingly serve as the "brain" of embodied AI, instructing robots to interact with the physical world. In such embodied settings, a central capability is the use of physical tools, which underpins MLLMs' ability to assist humans in real-world tasks.
arXiv:2610.14020v1 Announce Type: cross Abstract: Self-driving laboratories (SDLs) aim to automate the full experimental loop, from scientific decision-making to physical execution. Ideally, AI-gener...
ReactHuman is a physics‑grounded benchmark that tests whether multimodal large language models (MLLMs) can make immediate, safety‑critical decisions in simulated humanoid scenarios involving sudden household hazards. The benchmark includes 17 event families, over 1,000 reproducible scenes generated from 240 Hz rigid‑body simulation, and a five‑metric suite evaluating reactions on reasonableness, safety, and physical grounding. Evaluation of seven MLLMs reveals that reactive safety remains unsolved, with models frequently mishandling hazards, relying on appearance over motion, and missing key interception points.
The paper introduces VERID, a semantic digital twin for laboratory workcells that bridges AI agent reasoning with the physical realities of wet labs. VERID offers queryable introspection, time‑compressed simulation, trace persistence, and a safe execution framework, enabling agents to detect errors, recover mid‑run, and validate protocols in silico. Three case studies with dye and water mixtures demonstrate VERID’s ability to pre‑emptively catch critical errors, recover from failures, and provide structured multimodal data for downstream analysis and agent training.
arXiv:2606. 11324v1 Announce Type: cross Abstract: We introduce Embodied-R1.
arXiv:2610.18836v2 Announce Type: replace Abstract: Vision-Language Models (VLMs) have achieved strong performance in visual perception and spatial reasoning, leading to growing interest in using the...
arXiv:2605. 07306v2 Announce Type: replace-cross Abstract: Biological laboratory automation can reduce repetitive manual work and improve reproducibility, but reliable embodied execution in wet-lab environments remains challenging.
arXiv:2606. 13578v1 Announce Type: cross Abstract: Scientific laboratories increasingly rely on AI systems to reason about experiments, but the physical act of doing science remains largely outside their reach.
arXiv:2508.05427v2 Announce Type: replace Abstract: Large language models (LLMs) are beginning to reshape how organic-synthesis workflows are represented, queried, planned, and connected to experimen...