Towards Automated Ontology Generation from Unstructured Text: A Multi-Agent LLM Approach

2026-04-24

AI Summary

Automatically generating high-quality formal ontologies from unstructured text remains challenging, as existing large language model (LLM)-based approaches are often hindered by ambiguous design, structural redundancy, and ineffective repair mechanisms. This work proposes a planning-first, artifact-driven multi-agent paradigm for ontology generation, decomposing the task into a collaborative workflow among four specialized roles: domain expert, manager, coder, and quality assurer. The framework integrates heterogeneous LLM-based review, SPARQL competency assessment, and retrieval-augmented generation to iteratively refine ontological artifacts. Compared to single-agent baselines, the proposed approach substantially improves the structural quality and auditability of generated ontologies while moderately enhancing their query usability.

Technology Category

• Knowledge Representation and Reasoning: Ontologies
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Automatically generating formal ontologies from unstructured natural language remains a central challenge in knowledge engineering. While large language models (LLMs) show promise, it remains unclear which architectural design choices drive generation quality and why current approaches fail. We present a controlled experimental study using domain-specific insurance contracts to investigate these questions. We first establish a single-agent LLM baseline, identifying key failure modes such as poor Ontology Design Pattern compliance, structural redundancy, and ineffective iterative repair. We then introduce a multi-agent architecture that decomposes ontology construction into four artifact-driven roles: Domain Expert, Manager, Coder, and Quality Assurer. We evaluate performance across architectural quality (via a panel of heterogeneous LLM judges) and functional usability (via competency question driven SPARQL evaluation with complementary retrieval augmented generation based assessment). Results show that the multi-agent approach significantly improves structural quality and modestly enhances queryability, with gains driven primarily by front-loaded planning. These findings highlight planning-first, artifact-driven generation as a promising and more auditable path toward scalable automated ontology engineering.

Problem

• ontology generation
• unstructured text
• knowledge engineering
• large language models
• ontology design patterns

Innovation

• multi-agent LLM
• ontology generation
• artifact-driven design
• structured planning
• knowledge engineering