Minimizing Hyperbolic Embedding Distortion with LLM-Guided Hierarchy Restructuring

2025-11-16

AI Summary

Low branching factor and multiple inheritance in original hierarchical structures cause significant distortion in hyperbolic embeddings. Method: We propose the first LLM-based semantic-preserving hierarchical restructuring method, which—via carefully designed prompting—rewrites input hierarchies into optimized forms featuring high branching factor and single inheritance, while preserving semantic relationships and generating interpretable reconstruction rationales. Hyperbolic embedding quality is treated as an implicit optimization objective, balancing geometric fidelity and knowledge engineering practicality. Results: Experiments across 16 diverse real-world hierarchies demonstrate that the restructured hierarchies consistently and statistically significantly improve hyperbolic embedding performance across standard metrics—including precision@k, hit rate (HR), and distortion—validating both the effectiveness and generalizability of LLM-driven structural optimization.

Technology Category

• Machine Learning: Structured Learning
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Hyperbolic geometry is an effective geometry for embedding hierarchical data structures. Hyperbolic learning has therefore become increasingly prominent in machine learning applications where data is hierarchically organized or governed by hierarchical semantics, ranging from recommendation systems to computer vision. The quality of hyperbolic embeddings is tightly coupled to the structure of the input hierarchy, which is often derived from knowledge graphs or ontologies. Recent work has uncovered that for an optimal hyperbolic embedding, a high branching factor and single inheritance are key, while embedding algorithms are robust to imbalance and hierarchy size. To assist knowledge engineers in reorganizing hierarchical knowledge, this paper investigates whether Large Language Models (LLMs) have the ability to automatically restructure hierarchies to meet these criteria. We propose a prompt-based approach to transform existing hierarchies using LLMs, guided by known desiderata for hyperbolic embeddings. Experiments on 16 diverse hierarchies show that LLM-restructured hierarchies consistently yield higher-quality hyperbolic embeddings across several standard embedding quality metrics. Moreover, we show how LLM-guided hierarchy restructuring enables explainable reorganizations, providing justifications to knowledge engineers.

Problem

• Optimizes hyperbolic embeddings by restructuring hierarchies with LLMs
• Addresses embedding quality tied to hierarchy structure like branching factor
• Enables explainable reorganization of knowledge graphs for better embeddings

Innovation

• LLM-guided hierarchy restructuring for hyperbolic embeddings
• Prompt-based transformation using known hyperbolic embedding criteria
• Explainable reorganizations with justifications for knowledge engineers