Learning Chain Of Thoughts Prompts for Predicting Entities, Relations, and even Literals on Knowledge Graphs

2026-04-14

AI Summary

This work addresses the limited generalization of existing knowledge graph embedding models when confronted with unseen entities, relations, and literals, particularly in dynamic and heterogeneous settings. The authors propose the first approach that integrates chain-of-thought prompt learning into knowledge graph reasoning, reframing link prediction as a prompt optimization problem grounded in large language models. By employing Bayesian optimization (via the MIPRO algorithm), the method searches for string-level prompts that serve as interpretable scoring functions for triples—requiring no gradient updates and adapting to new tasks with fewer than 30 samples. It further supports complex OWL class expression reasoning. Experimental results demonstrate consistent improvements, with mean reciprocal rank (MRR) exceeding state-of-the-art models by over 5% across multiple benchmarks and achieving Jaccard similarity above 88% on OWL reasoning tasks, thereby significantly enhancing both generalization and interpretability.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Knowledge graph embedding (KGE) models perform well on link prediction but struggle with unseen entities, relations, and especially literals, limiting their use in dynamic, heterogeneous graphs. In contrast, pretrained large language models (LLMs) generalize effectively through prompting. We reformulate link prediction as a prompt learning problem and introduce RALP, which learns string-based chain-of-thought (CoT) prompts as scoring functions for triples. Using Bayesian Optimization through MIPRO algorithm, RALP identifies effective prompts from fewer than 30 training examples without gradient access. At inference, RALP predicts missing entities, relations or whole triples and assigns confidence scores based on the learned prompt. We evaluate on transductive, numerical, and OWL instance retrieval benchmarks. RALP improves state-of-the-art KGE models by over 5% MRR across datasets and enhances generalization via high-quality inferred triples. On OWL reasoning tasks with complex class expressions (e.g., $\exists hasChild.Female$, $\geq 5 \; hasChild.Female$), it achieves over 88% Jaccard similarity. These results highlight prompt-based LLM reasoning as a flexible alternative to embedding-based methods. We release our implementation, training, and evaluation pipeline as open source: https://github.com/dice-group/RALP .

Problem

• knowledge graph
• link prediction
• unseen entities
• literals
• generalization

Innovation

• Chain-of-Thought Prompting
• Knowledge Graph Reasoning
• Prompt Learning
• Bayesian Optimization
• Large Language Models