The article reviews methods for assessing large language model (LLM) based AI agents in materials synthesis, focusing on their integration with experimental tools. It outlines evaluation strategies—including knowledge, reasoning, tool‑use, and closed‑loop benchmarks—and applies them to atomic layer deposition (ALD) as a case study. A practical framework for evaluating LLMs in this context is also presented.
arXiv:2608. 15776v1 Announce Type: cross Abstract: Despite the powerful multi-scale modeling methods and high-throughput infrastructures established in the materials community, real material computation workflows remain fragmented and heavily manual, requiring researchers to constantly bridge software tools, data analysis, and intermediate decisions.
SynAgent is a framework that uses large language model agents to run autonomous experiments while building an explicit, revisable understanding of the synthesis process. Unlike traditional black‑box optimizers, SynAgent generates analysis skills on the fly and reasons multimodally over data such as X‑ray diffraction patterns and electron micrographs. In an 18‑experiment campaign on LiCoO₂ thin‑film deposition, SynAgent produced highly crystalline films and uncovered a sharp temperature threshold and optimal growth window (650–690 °C) for crystallization.
arXiv:2512. 11935v2 Announce Type: replace Abstract: Agentic AI systems increasingly connect large language models (LLMs) to external scientific tools, yet whether and when tool access improves prediction accuracy remains uncharacterized.
The article introduces the concept of agentic programs—scientific software that blends deterministic algorithms with bounded large‑language‑model (LLM) judgment, task‑specific verification, episodic maturation, and full delegation in production. It argues that recent LLM‑based agents enable this new form of computational materials science software. The authors illustrate the idea with DeMARS, an agentic program designed to build atomistic models from experimentally measured disordered crystal structures.
arXiv:2508.05427v2 Announce Type: replace Abstract: Large language models (LLMs) are beginning to reshape how organic-synthesis workflows are represented, queried, planned, and connected to experimen...
El Agente Potente is an agentic system that integrates typed execution graphs and a coding mode to facilitate machine‑learning interatomic potential (MLIP) driven atomistic simulations. Typed execution graphs offer structured, provenance‑aware workflows where large language models handle planning and routing while deterministic Python code performs scientific computation and validation. The coding agent builds customized workflows for tasks needing procedural flexibility, invoking existing Potente functions for supported calculations. The system is demonstrated across materials discovery, energy‑landscape exploration, adsorption, and catalytic reaction workflows, with benchmarks on reproducibility and LLM token cost.
The paper introduces AutoMat, a benchmark designed to test large language model (LLM) coding agents on their ability to reproduce claims from computational materials science. AutoMat presents three challenges: reconstructing underspecified procedures, navigating specialized toolchains, and assessing whether the evidence supports a claim. Experiments show that current LLM agents achieve low success rates, with the best setting reaching only 53%, and failures stem mainly from incomplete procedures, methodological deviations, and execution fragility.
MAESTRO is a large language model agent that automates the full screening pipeline for metal‑organic frameworks (MOFs). It parses extensive MOF literature, links publications to crystal structures, curates a computation‑ready database, and then applies a progressively more expensive computational strategy to identify promising candidates. The identified materials for wet flue gas separation come from unrelated studies, demonstrating the agent’s ability to uncover high‑performance materials across domains.
The paper introduces a new benchmark that evaluates large language models (LLMs) on their agentic mathematical reasoning rather than just final answers. It aligns problem‑solving behaviors with a taxonomy of reusable mathematical atomic capabilities and includes planning, action, and feedback tasks in both textual and multimodal settings. Experiments show that models with similar end‑to‑end accuracy can have very different agentic profiles, highlighting the importance of process‑level evaluation.
arXiv:2607. 11526v1 Announce Type: cross Abstract: Material property prediction (MPP) infers key properties from chemical composition and structure, accelerating the discovery and optimization of novel materials.
arXiv:2506. 05616v3 Announce Type: replace Abstract: We aim at designing language agents with greater autonomy for crystal materials discovery.
arXiv:2603. 13191v2 Announce Type: replace-cross Abstract: While large language models (LLMs) have transformed AI agents into proficient executors of computational materials science, performing a hundred simulations does not make a researcher.