arXiv AI

When CQs Go Wrong: Challenges in CQ Verification with OE-Assist

arXiv:2606. 24619v1 Announce Type: new Abstract: Competency Questions (CQs) are the central component of CQ-verification, an established process in which an ontology is evaluated against a set of natural language questions to determine whether the intended purpose of the ontology has been properly modelled.

arXiv Computation and Language
Sep 24

UniDataAgent: An Ontology-Grounded Agent for Enterprise Question-to-Report Automation

UniDataAgent (UniDataAgent) is an ontology‑grounded system designed to automate enterprise question‑to‑report tasks while preserving organization‑specific semantics. It separates semantic acquisition from online execution, with an Ontology Acquisition and Validation (OAV) stage that builds versioned ontologies from metadata, business knowledge, and expert input, and a Question‑to‑Report Execution (QRE) stage that retrieves semantic contracts, coordinates skills and data tools, validates results, and produces evidence‑linked reports. In a deployment across 27 enterprise tables and thousands of metric types, ontology construction took a few hours versus a week manually, and report generation took minutes versus several working days, achieving 95.0% strict accuracy on real business questions compared to 72.5% for document RAG.

By Yutai Duan, Yahui Zhao, Zhangti Li, Yu Ma, Zhenfeng Qi, Shaoyang Yuan, Jing Fan, Jie Liu
arXiv AI
Aug 28

A Task-Centric Ontology and Deterministic Domain Rules as a Verifiable Core for AI-Assisted Chemistry Problem Solving

The paper introduces ChemOntoRule, a symbolic core designed to aid AI in solving school‑level chemistry problems. It uses a task‑centric ontology built around the specific concepts and procedures needed for a defined set of problems, combined with deterministic Python rules for electronic structure, periodic trends, oxidation states, and related reasoning patterns. Evaluated on 300 human‑authored problems, the system matched 296 reference answers (98.67%), with the ontology‑driven rules covering 269 problems and achieving 98.88% accuracy.

By Ibrokhimsho Abduchaborov
arXiv AI
2d ago

Ontology-Grounded, Reasoner-Verified Benchmarks for Evaluating LLM Reasoning in Scientific AI

The paper introduces a pipeline that automatically creates ontology‑grounded multiple‑choice question benchmarks for evaluating large language models (LLMs) on logical reasoning tasks in scientific AI. By using OWL 2 ontologies, correct answers are guaranteed by design and distractors are generated and formally verified as incorrect through an OWL reasoner. Experiments on three ontologies—Pizza, PMDco, and DOID—yielded 112, 2,491, and 15,216 MCQs, respectively, with high natural‑language quality and challenging zero‑shot performance for six LLMs.

By Nishtha N. Vaidya, Stephan Grimm, Thomas Hubauer, Thomas A. Runkler
arXiv Machine Learning
Sep 14

GLaMoR: Consistency Checking of OWL Ontologies using Graph Language Models

The paper introduces GLaMoR, a reasoning pipeline that converts OWL ontologies into graph-structured data and applies a Graph Language Model (GLM) for consistency checking. It addresses the challenge of verifying ontology consistency, especially for large ontologies where classical reasoners become computationally expensive. Experiments on NCBO BioPortal ontologies show that GLaMoR achieves 95% accuracy and is 20 times faster than traditional reasoners.

By Justin M\"ucke, Ansgar Scherp