A Graph Perspective to Probe Structural Patterns of Knowledge in Large Language Models

2025-05-25

AI Summary

The organization of implicit knowledge in large language models (LLMs) remains poorly understood, particularly due to the lack of interpretable, quantitative measures for knowledge distribution. Method: We propose a knowledge-graph-based framework to quantify knowledge strength at both triple and entity levels, revealing—through empirical analysis—a previously unobserved graph-structural homophily phenomenon within LLMs: high-knowledge-strength entities preferentially connect to other high-knowledge-strength entities. Leveraging this insight, we design a local-neighborhood-aware knowledge estimator for fine-grained identification of knowledge blind spots. Based on these findings, we introduce a knowledge-aware fine-tuning strategy. Contribution/Results: Our approach significantly improves performance across multiple downstream tasks. It establishes a novel paradigm for LLM knowledge editing, enhancement, and interpretability analysis by grounding knowledge representation in principled graph-theoretic principles.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large language models have been extensively studied as neural knowledge bases for their knowledge access, editability, reasoning, and explainability. However, few works focus on the structural patterns of their knowledge. Motivated by this gap, we investigate these structural patterns from a graph perspective. We quantify the knowledge of LLMs at both the triplet and entity levels, and analyze how it relates to graph structural properties such as node degree. Furthermore, we uncover the knowledge homophily, where topologically close entities exhibit similar levels of knowledgeability, which further motivates us to develop graph machine learning models to estimate entity knowledge based on its local neighbors. This model further enables valuable knowledge checking by selecting triplets less known to LLMs. Empirical results show that using selected triplets for fine-tuning leads to superior performance.

Problem

• Investigates structural knowledge patterns in LLMs via graphs
• Analyzes knowledge relation to graph properties like node degree
• Develops graph models to estimate entity knowledge from neighbors

Innovation

• Graph perspective analyzes LLM knowledge structure
• Graph ML models estimate entity knowledge locally
• Selected triplets enhance fine-tuning performance