arXiv AI By Angel Yanguas-Gil

Evaluating LLM-based AI agents integrated with materials synthesis tools: the case of atomic layer deposition

Read the original on arXiv AI →

The article reviews methods for assessing large language model (LLM) based AI agents in materials synthesis, focusing on their integration with experimental tools. It outlines evaluation strategies—including knowledge, reasoning, tool‑use, and closed‑loop benchmarks—and applies them to atomic layer deposition (ALD) as a case study. A practical framework for evaluating LLMs in this context is also presented.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Aug 18

ALKEMIE Agent: an autonomous platform for computational materials design

arXiv:2608. 15776v1 Announce Type: cross Abstract: Despite the powerful multi-scale modeling methods and high-throughput infrastructures established in the materials community, real material computation workflows remain fragmented and heavily manual, requiring researchers to constantly bridge software tools, data analysis, and intermediate decisions.

By Hongfu Huang, Yuzhe Li, Ao Xu, Bo Liu, Changrui Wang, Kan Tang, Ning Yang, Shengxian Liu, Hanyu Liu, Pengpeng Zhang, Linggang Zhu, Fengkai Liu, Yichen Lu, Tong Zhao, Naihua Miao, Jian Zhou, Zhimei Sun
arXiv AI
Sep 17

Hypothesis-Driven Autonomous Materials Synthesis with Multimodal LLM Agents

SynAgent is a framework that uses large language model agents to run autonomous experiments while building an explicit, revisable understanding of the synthesis process. Unlike traditional black‑box optimizers, SynAgent generates analysis skills on the fly and reasons multimodally over data such as X‑ray diffraction patterns and electron micrographs. In an 18‑experiment campaign on LiCoO₂ thin‑film deposition, SynAgent produced highly crystalline films and uncovered a sharp temperature threshold and optimal growth window (650–690 °C) for crystallization.

By Izumi Takahara, Kazunori Nishio, Akira Aiba, Shigeru Kobayashi, Takao Nakajima, Taro Hitosugi, Teruyasu Mizoguchi
arXiv AI
Oct 2

CompMat-Bench: Benchmarking AI Agents for Computational Materials Science

CompMat-Bench is a new benchmark comprising 94 tasks drawn from recent computational materials science studies, designed to evaluate AI agents on realistic research steps without requiring costly simulations during testing. The benchmark pre‑reproduces inputs and outputs to provide ground truth, allowing agents to be graded with fixed rules rather than an LLM judge. It supports both single tasks and multi‑step workflows, with varying levels of methodological guidance, and shows that while agents can achieve high pass rates on individual tasks, performance drops in longer workflows or with reduced guidance, often due to scientific rather than software errors.

By Chenmu Zhang, Levi Felix, Jun-Jie Zhang, Xingfu Li, Xuelian Jiang, Tao Jiang, Subhendu Mishra, Xixi Qin, Boris Yakobson