Language Models as Ontology Encoders

2025-07-18

AI Summary

Existing ontology embedding methods suffer from an inherent tension between semantic and structural representation: geometric models neglect textual semantics, while language models struggle to capture logical structure. This paper proposes OnT—the first ontology embedding framework that deeply integrates pretrained language models with hyperbolic geometric modeling. OnT is the first to align the hierarchical semantics of the EL description logic and the textual descriptions of OWL ontologies within hyperbolic space. Leveraging contrastive learning and structure-aware fine-tuning, OnT jointly encodes class hierarchies, logical axioms, and contextual semantics. Evaluated on four standard ontology benchmark datasets, OnT achieves significant improvements over state-of-the-art methods in both axiom prediction and logical reasoning tasks. Furthermore, it successfully transfers to construct SNOMED CT, demonstrating strong generalizability and practical utility.

Technology Category

• Knowledge Representation and Reasoning: Ontologies
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

OWL (Web Ontology Language) ontologies which are able to formally represent complex knowledge and support semantic reasoning have been widely adopted across various domains such as healthcare and bioinformatics. Recently, ontology embeddings have gained wide attention due to its potential to infer plausible new knowledge and approximate complex reasoning. However, existing methods face notable limitations: geometric model-based embeddings typically overlook valuable textual information, resulting in suboptimal performance, while the approaches that incorporate text, which are often based on language models, fail to preserve the logical structure. In this work, we propose a new ontology embedding method OnT, which tunes a Pretrained Language Model (PLM) via geometric modeling in a hyperbolic space for effectively incorporating textual labels and simultaneously preserving class hierarchies and other logical relationships of Description Logic EL. Extensive experiments on four real-world ontologies show that OnT consistently outperforms the baselines including the state-of-the-art across both tasks of prediction and inference of axioms. OnT also demonstrates strong potential in real-world applications, indicated by its robust transfer learning abilities and effectiveness in real cases of constructing a new ontology from SNOMED CT. Data and code are available at https://github.com/HuiYang1997/OnT.

Problem

• Existing ontology embeddings ignore text or logical structure
• Need method combining text and logical hierarchy preservation
• Current approaches underperform in prediction and inference tasks

Innovation

• Tunes PLM via hyperbolic geometric modeling
• Incorporates textual labels and logical structure
• Outperforms baselines in prediction and inference