Ontology Generation using Large Language Models

2025-03-07

AI Summary

To address the complexity, time consumption, and error-proneness of ontology engineering, this paper proposes an automated OWL ontology generation method leveraging large language models (LLMs). The approach introduces two novel prompting strategies—Memoryless CQbyCQ and Ontogenia—that enable end-to-end generation of OWL ontology drafts from user stories and competency questions (CQs). Furthermore, we establish a structured, three-dimensional evaluation framework assessing syntactic correctness, semantic fidelity, and logical completeness. Experimental results on a benchmark comprising 10 domain ontologies and 100 CQs demonstrate that Ontogenia, when paired with OpenAI’s o1-preview model, significantly outperforms existing LLM-assisted methods. The generated ontologies meet engineer-ready quality standards and surpass the output quality typically produced by novice ontology engineers.

Technology Category

• Knowledge Representation and Reasoning: Ontologies
• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

The ontology engineering process is complex, time-consuming, and error-prone, even for experienced ontology engineers. In this work, we investigate the potential of Large Language Models (LLMs) to provide effective OWL ontology drafts directly from ontological requirements described using user stories and competency questions. Our main contribution is the presentation and evaluation of two new prompting techniques for automated ontology development: Memoryless CQbyCQ and Ontogenia. We also emphasize the importance of three structural criteria for ontology assessment, alongside expert qualitative evaluation, highlighting the need for a multi-dimensional evaluation in order to capture the quality and usability of the generated ontologies. Our experiments, conducted on a benchmark dataset of ten ontologies with 100 distinct CQs and 29 different user stories, compare the performance of three LLMs using the two prompting techniques. The results demonstrate improvements over the current state-of-the-art in LLM-supported ontology engineering. More specifically, the model OpenAI o1-preview with Ontogenia produces ontologies of sufficient quality to meet the requirements of ontology engineers, significantly outperforming novice ontology engineers in modelling ability. However, we still note some common mistakes and variability of result quality, which is important to take into account when using LLMs for ontology authoring support. We discuss these limitations and propose directions for future research.

Problem

• Automating ontology generation using Large Language Models
• Evaluating new prompting techniques for ontology development
• Assessing quality and usability of LLM-generated ontologies

Innovation

• LLMs generate OWL ontologies from user stories.
• Memoryless CQbyCQ and Ontogenia prompting techniques introduced.
• Multi-dimensional evaluation for ontology quality and usability.