Can LLMs Faithfully Solve Ontology Engineering Tasks? A Mechanistic Interpretability Study

The paper introduces MIOE, a causal‑abstraction benchmark designed to probe how large language models (LLMs) handle ontology engineering tasks such as entity typing and subsumption prediction. Using this benchmark, the authors evaluate several mechanistic interpretability methods—including SAE, DBM, and DAS—on Qwen3‑8B and Qwen3‑4B models, finding that distributed subspace interventions perform best but still mix with auxiliary class‑specific information. The study shows that while ontological predictions are accessible at a mechanistic level, current techniques do not yet isolate a clean, disentangled causal mechanism for ontological knowledge.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• benchmarks
• safety