AI agents

Tool use, function calling, orchestration and the protocols that let models act rather than only answer.

15,743 stories · RSS feed

arXiv AI
Sep 30

EngiWorld: What Can Frontier Agents Deliver in Professional Engineering Environments?

EngiWorld is a new benchmark that tests autonomous agents across the full engineering design loop, covering 1,301 expert‑curated tasks in six domains (CAD, CAE, CAM, BIM, EDA, and 3D visualization) and 26 professional software platforms with both GUI and CLI interfaces. It introduces an artifact‑centric evaluation method that programmatically verifies geometric validity, physical feasibility, and rule compliance of both final and intermediate artifacts, scoring tasks continuously rather than with binary success. Initial tests of seven frontier models show a large capability gap, with the best model scoring only 44.3 on the EngiScore and just 3.6% of multi‑software attempts succeeding.

By Hongcheng Gao, Hailong Qu, Yu Lei, Henghui Sun, Haoyang Li, Yipeng Wei, Naihao Xue, Xiaohan Yu, Zhuo Tao, Yihe Zang, Yajiao Wang, Jingyi Tang, Yi Li, Jingjing Zhou, Jie Luo, Bohan Zeng, Chengyu Shen, Hao Jiang, Chong Chen, Bowen Qu, Olive Huang, Zeqiang Wang