PHMForge is an evaluation environment that tests large language model agents on industrial prognostics tasks using the Model Context Protocol (MCP). It provides 99 SME-authored scenarios across eight asset classes, 39 MCP-native tools wrapping published PHM algorithms, and deterministic evaluators that separate protocol fluency, reasoning, instrumentation, and tool use. The benchmark shows that the best agent configuration achieves 80.8% pass@1, with orchestration and tool sequencing errors as the main failure modes, and demonstrates the limitations of static retrieval for prognostic computation.
By Yusheng Li, Tianjun Feng, Yunfeng Chen, Chun-Yi Tsai, Yihan Sun, Ayan Das, Kaoutar El Maghraoui, Shuxin Lin, Dhaval Patel
arXiv:2605.02346v2 Announce Type: replace-cross
Abstract: Operational technology (OT) devices run safety-critical physical processes, yet their security testing remains manual and expert-intensive. A...
By Adel ElZemity, Budi Arief, Shujun Li, George Oikonomou, James Pope
arXiv:2609.10494v1 Announce Type: new
Abstract: Enterprises deploy systems, not checkpoints. Usable capability depends jointly on weights, serving route, precision, output contract, and harness, yet...
By Blake Stenstrom, Charangan Vasantharajan, Brian Sathianathan
arXiv:2608. 11241v1 Announce Type: new Abstract: Deploying LLM agents into industrial recommender operations exposes a three-way tension we frame as the autonomy-determinism-efficiency trilemma: general autonomy (interpreting operator intent, generating glue code zero-shot), industrial determinism (schema-conforming feature extraction, non-crashing A/B, zero compliance-path hallucination), and end-to-end efficiency.
By Dongyang Ao, Kaixiang Fang, Shijie Xu
arXiv:2607. 17269v1 Announce Type: new Abstract: Large language models encode world models implicitly in neural weights, which exposes four structural risks in high-precision domains such as medicine and finance: hallucination, frozen knowledge, poor explainability, and poor modifiability.
By Zhanbo Li, Shifeng Wu, Xiangjin Meng, Wenjie Cai
Enterprises deploy systems, not checkpoints. Usable capability depends jointly on weights, serving route, precision, output contract, and harness, yet all 18 audited benchmarks score advertised model...