The paper introduces a hierarchy of four increasingly strict faithfulness criteria—discrimination, logical admissibility, monotonic logical faithfulness, and probabilistic logical faithfulness—for evaluating knowledge base completion models, particularly when the target is a description logic knowledge base. It demonstrates that ranking accuracy alone does not guarantee logical faithfulness and shows that current embedding models fail to satisfy any of the criteria across the hierarchy. The authors provide a formal grounding for the strongest criterion using relative model counts and evaluate several models on εL ontologies, revealing gaps between performance metrics and logical correctness.
arXiv:2607. 08017v1 Announce Type: cross Abstract: Large-Language Models (LLMs) can be prone to flawed and unfaithful reasoning that decoding strategies like Self-Consistency (SC) fail to detect as they evaluate only final-answer agreement while ignoring the logical validity of intermediate steps.
arXiv:2608. 15147v1 Announce Type: new Abstract: Machine intelligence has conquered the symbolic world but stalled at the physical one.
The paper introduces a pipeline that automatically creates ontology‑grounded multiple‑choice question benchmarks for evaluating large language models (LLMs) on logical reasoning tasks in scientific AI. By using OWL 2 ontologies, correct answers are guaranteed by design and distractors are generated and formally verified as incorrect through an OWL reasoner. Experiments on three ontologies—Pizza, PMDco, and DOID—yielded 112, 2,491, and 15,216 MCQs, respectively, with high natural‑language quality and challenging zero‑shot performance for six LLMs.
arXiv:2604. 17621v2 Announce Type: replace Abstract: Many real-world questions appear deceptively simple yet implicitly demand two capabilities: (i) systematic coverage of a bounded knowledge universe and (ii) compositional set-based reasoning over that universe, a phenomenon we term "the tip of the iceberg.
Baobab compiles an OWL 2 DL (ΣROIQ) ontology with a finite ABox into a Sentential Decision Diagram (SDD), saturating a propositional core and instantiating remaining DL features over the active domain. The resulting evidence‑conditioned weighted model count trains a perception network to recognize real images under partial ABox supervision, enabling a CNN to recover latent ontology concepts that an independent perception would miss. When supervision allows multiple ontology‑consistent completions, Baobab’s mixture indexed by query justifications represents the calibrated posterior, achieving Bayes‑optimal performance on a real‑image MNIST task where single‑WMC and learned mixtures fail, thereby characterizing and mitigating reasoning shortcuts in a non‑Horn description logic.
arXiv:2608. 12961v1 Announce Type: new Abstract: The OWL 2 EL profile is used in some of the largest production ontologies, including the Gene Ontology and SNOMED CT.
arXiv:2606. 18557v1 Announce Type: new Abstract: A rule-based logic solver resolves every instance in our benchmark in under 50 microseconds with 100% accuracy; the best frontier language model reaches 65% at best and drops to 23.
arXiv:2606. 15246v1 Announce Type: cross Abstract: Provenance-enhanced statements of the form "according to $X$, $\varphi$" are pervasive in contemporary knowledge graphs, especially in domains where graph content primarily represents claims, interpretations, and hypotheses (\emph{capta}) rather than observer-independent facts (\emph{data}).
arXiv:2608.23370v1 Announce Type: new Abstract: Large Language Models (LLMs) recognise patterns but do not natively track the path of exclusions that a coherent discourse demands. When an input rests...
arXiv:2604. 11996v2 Announce Type: replace-cross Abstract: Should we trust Large Language Models (LLMs) with high accuracy?
arXiv:2609.38420v1 Announce Type: cross Abstract: We introduce Type-6 logic, a variant of dynamic epistemic logic augmented with two operators (uncertainty and recurrence), designed to model the infe...
arXiv:2609.06127v1 Announce Type: cross Abstract: Retrieval-augmented generation is usually formulated as a statistical information-retrieval problem. Graph-based variants add relational structure, b...