DiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules

DiagnosticIQ is a benchmark that tests large language models (LLMs) on translating engineer‑written symbolic rules into maintenance actions for industrial assets. It contains 6,690 multiple‑choice questions derived from 118 expert rule‑action pairs across 16 asset types, and includes five variants that probe different failure modes. The benchmark evaluates 29 LLMs and four embedding baselines, showing that while top models perform near the frontier, they exhibit brittleness under distractor expansion and structural perturbations.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• rag
• benchmarks