Assessing the Capability of Large Language Models for Domain-Specific Ontology Generation

2025-04-24

AI Summary

Traditional ontology engineering relies heavily on manual effort and suffers from poor generalizability across domains. Method: This study systematically evaluates the applicability and cross-domain generalization capability of large language models (LLMs) for automated ontology generation, proposing a capability-question (CQ)-driven prompt engineering framework. We empirically assess DeepSeek and o1-preview across six real-world ontology engineering projects using 95 curated CQs. Contribution/Results: We present the first empirical validation that structurally reasoning-capable LLMs can consistently generate high-quality ontologies across diverse domains—demonstrating domain-agnostic performance. Both models reliably perform ontology schema extraction, class/property identification, and logical axiom generation. This significantly enhances automation, scalability, and reproducibility in ontology construction. Our work establishes a novel paradigm for general-purpose ontology engineering, reducing reliance on domain-specific expertise and manual curation while maintaining formal expressivity and semantic fidelity.

Technology Category

• Knowledge Representation and Reasoning: Ontologies
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Large Language Models (LLMs) have shown significant potential for ontology engineering. However, it is still unclear to what extent they are applicable to the task of domain-specific ontology generation. In this study, we explore the application of LLMs for automated ontology generation and evaluate their performance across different domains. Specifically, we investigate the generalizability of two state-of-the-art LLMs, DeepSeek and o1-preview, both equipped with reasoning capabilities, by generating ontologies from a set of competency questions (CQs) and related user stories. Our experimental setup comprises six distinct domains carried out in existing ontology engineering projects and a total of 95 curated CQs designed to test the models' reasoning for ontology engineering. Our findings show that with both LLMs, the performance of the experiments is remarkably consistent across all domains, indicating that these methods are capable of generalizing ontology generation tasks irrespective of the domain. These results highlight the potential of LLM-based approaches in achieving scalable and domain-agnostic ontology construction and lay the groundwork for further research into enhancing automated reasoning and knowledge representation techniques.

Problem

• Evaluate LLMs' ability to generate domain-specific ontologies
• Test generalizability of DeepSeek and o1-preview models across domains
• Assess scalability of LLM-based ontology construction methods

Innovation

• Utilizing LLMs for automated ontology generation
• Evaluating generalizability across multiple domains
• Leveraging competency questions for reasoning assessment