TRAIL: Joint Inference and Refinement of Knowledge Graphs with Large Language Models

2025-08-06

AI Summary

This work addresses the fragmentation between large language models (LLMs) and knowledge graphs (KGs), which leads to delayed knowledge updates, uninterpretable reasoning, and weak factual consistency. To this end, we propose a unified framework for joint reasoning and incremental learning, centered on a confidence-driven mechanism that integrates chain-of-thought reasoning, dynamic graph structure updates, and external memory interaction. The framework enables autonomous knowledge generation, credibility-aware validation, and redundancy pruning, supporting training-free real-time knowledge integration. Its key innovation lies in the deep synergy of symbolic and neural reasoning, establishing a closed-loop process wherein KG evolution and LLM inference mutually reinforce each other. Evaluated on multiple knowledge-intensive benchmarks, our method achieves 3–13% absolute accuracy gains over state-of-the-art approaches, significantly improving factual correctness, adaptability, and interpretability.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Recent advances in large language models (LLMs) have unlocked powerful reasoning and decision-making capabilities. However, their inherent dependence on static parametric memory fundamentally limits their adaptability, factual accuracy, and interpretability in knowledge-intensive scenarios. Knowledge graphs (KGs), as structured repositories of explicit relational knowledge, offer a promising approach for augmenting LLMs with external, interpretable memory. Nevertheless, most existing methods that combine LLMs with KGs treat reasoning and knowledge updating as separate processes, resulting in suboptimal utilization of new information and hindering real-time updates. In this work, we propose TRAIL: a novel, unified framework for Thinking, Reasoning, And Incremental Learning that couples joint inference and dynamic KG refinement with large language models. TRAIL enables LLM agents to iteratively explore, update, and refine knowledge graphs during the reasoning process, employing a confidence-driven mechanism for the generation, validation, and pruning of new facts. This plug-and-play architecture facilitates seamless integration with various LLMs, supporting continual adaptation without the need for retraining. Extensive experiments on multiple benchmarks demonstrate that TRAIL outperforms existing KG-augmented and retrieval-augmented LLM baselines by 3% to 13%. More importantly, these results represent a significant step toward developing adaptive, memory-augmented language models capable of continual learning and reliable, transparent reasoning.

Problem

• Enhance LLM adaptability with dynamic knowledge graphs
• Integrate joint inference and KG refinement in real-time
• Improve factual accuracy via confidence-driven fact validation

Innovation

• Joint inference and dynamic KG refinement
• Confidence-driven fact generation and pruning
• Plug-and-play continual adaptation without retraining