arXiv AI October 7, 2026 By Ardalan Aryashad, Yan Jin

A Validated Dataset and Benchmark for Coherent Multi-Diagram SysML Models

The paper introduces SEMAADB, a dataset comprising 3,000 engineering contexts and 15,000 SysML diagrams, each context containing five interconnected views (Requirement, Block Definition, Activity, State Machine, and Sequence). The authors verified diagram consistency and created a 100-context human‑verified benchmark. They evaluated three language models on diagram repair and cross‑diagram update tasks, finding that while syntax repair is largely solved, semantic repair and cross‑diagram consistency remain challenging.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Hugging Face Trending Papers

Jun 22

Automated Semantic Fault Localization in SysML v2: A Human-in-the-Loop Framework Using Knowledge-Graph Augmented LLMs

SysML v2's textual syntax enables compiler-based validation of model structure and language conformance. However, semantic mistakes that preserve syntactic validity but violate domain rules cannot be detected through compilers.

llms fine-tuning

arXiv AI

Jul 28

ERUnderstand: Evaluating Vision-Language Models on Structured ER Diagrams

arXiv:2607. 24707v1 Announce Type: new Abstract: Entity-Relationship Diagrams (ERDs) are central to conceptual database design, yet they are typically available only as rendered images rather than machine-readable schemas, limiting AI-assisted database engineering.

By Ali Ansari, Yasmin Mohammadi, Farnoush Nili, Parsa Esmaeilkhani, Longin Jan Latecki, Eduard Dragut

llms multimodal benchmarks

arXiv AI

Aug 13

Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams

arXiv:2608. 12262v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have been growing the capability for scientific writing and collaboration.

By Weihao Bo, Shan Zhang, Yanpeng Sun, Jie Liu, Yongke Yao, Jinhao Du, Wei He, Kai Zou, Zechao Li, Jingdong Wang

llms agents nlp multimodal benchmarks

arXiv AI

Aug 7

MermaidSeqBench: An Evaluation Benchmark for NL-to-Mermaid Sequence Diagram Generation

arXiv:2511. 14967v3 Announce Type: replace-cross Abstract: Large language models (LLMs) have shown great promise in generating structured diagrams from natural language descriptions, particularly Mermaid sequence diagrams for software engineering.

By Basel Shbita, Farhan Ahmed, Chad DeLuca

llms benchmarks

arXiv AI

Jul 29

Crystalis: Progressive Nucleation and Semantic Annealing for Coordinated Multi-View Visualization Generation

arXiv:2607. 24766v1 Announce Type: new Abstract: Large language models (LLMs) can generate individual charts, but coordinated multi-view visualizations (CMVs), where views share data flows and cross-view interactions, remain out of reach.

By Dazhen Deng, Zhaoping He, Xin Qian, Xiaotong Wang, Zi Ying, Yingcai Wu

llms agents benchmarks

arXiv Machine Learning

Aug 11

Math-Vision Diagrams: A Comprehensive Benchmark for Evaluating LLM Mathematical Diagram Generation Capabilities

arXiv:2608. 08964v1 Announce Type: new Abstract: The generation of mathematically precise diagrams from tex- tual prompts has emerged as a critical yet underexplored capability of Large Language Models (LLMs).

By Harish Kashyap, Kiran Byadarhaly, Sriram Chakaravarthy, Sanyukta Tuti, Aryan Mistry

llms diffusion benchmarks

One email a morning, machine-written

Related stories

Automated Semantic Fault Localization in SysML v2: A Human-in-the-Loop Framework Using Knowledge-Graph Augmented LLMs

ERUnderstand: Evaluating Vision-Language Models on Structured ER Diagrams

Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams

MermaidSeqBench: An Evaluation Benchmark for NL-to-Mermaid Sequence Diagram Generation

Crystalis: Progressive Nucleation and Semantic Annealing for Coordinated Multi-View Visualization Generation

Math-Vision Diagrams: A Comprehensive Benchmark for Evaluating LLM Mathematical Diagram Generation Capabilities