Quality Metrics for LLM-Generated Asset Administration Shells: A Perturbation-Based Evaluation Approach

The paper introduces a perturbation-based framework to evaluate quality metrics for large language model (LLM) generated Asset Administration Shells (AAS). By systematically degrading AAS outputs across multiple dimensions, the study identifies that exact property name matching and value-based recall, along with name-based F1 score, best reflect quality changes. Experiments on 6,400 AAS instances from 200 products across GPT‑4o‑mini, Qwen3, and DeepSeek‑R1 reveal how different perturbations affect metrics and highlight variations among model families and product segments.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms