arXiv AI

A Validated Dataset and Benchmark for Coherent Multi-Diagram SysML Models

The paper introduces SEMAADB, a dataset comprising 3,000 engineering contexts and 15,000 SysML diagrams, each context containing five interconnected views (Requirement, Block Definition, Activity, State Machine, and Sequence). The authors verified diagram consistency and created a 100-context human‑verified benchmark. They evaluated three language models on diagram repair and cross‑diagram update tasks, finding that while syntax repair is largely solved, semantic repair and cross‑diagram consistency remain challenging.

arXiv AI
Jul 28

ERUnderstand: Evaluating Vision-Language Models on Structured ER Diagrams

arXiv:2607. 24707v1 Announce Type: new Abstract: Entity-Relationship Diagrams (ERDs) are central to conceptual database design, yet they are typically available only as rendered images rather than machine-readable schemas, limiting AI-assisted database engineering.

By Ali Ansari, Yasmin Mohammadi, Farnoush Nili, Parsa Esmaeilkhani, Longin Jan Latecki, Eduard Dragut
arXiv Machine Learning
Jul 28

Benchmarking LLMs for Verilog Design Flows

arXiv:2607. 22759v1 Announce Type: cross Abstract: Large language models (LLMs) show promise in code generation, but their capabilities to produce correct, synthesizable hardware description language (HDL) code still remain to be properly benchmarked.

By Angshuman Chakravertty, Rahul Koshti, Buddhi Prakash Sharma, Vinay Chamola
arXiv AI
Sep 15

Natural-Language to SysMLv2 Translation via Conformance-Driven Iterative Refinement

The paper introduces a framework that translates natural‑language descriptions into SysMLv2 models using a generate‑check‑repair loop driven by a SysMLv2 conformance checker. By embedding the checker as an oracle, the system iteratively repairs generated models until they achieve zero conformance errors, ensuring they are deployable in industrial modeling environments. Evaluation on 151 prompts across four large language models shows the approach raises production‑conformance acceptance from 51.16% to 100%.

By Chance LaVoie, Eladio Andujar Lugo, Taylan G. Topcu, Levent Burak Kara