Efficient Knowledge Probing of Large Language Models by Adapting Pre-trained Embeddings

2025-08-08

AI Summary

Existing knowledge probing methods rely on forward passes through large language models (LLMs), incurring high computational cost and low efficiency. This paper proposes PEEK, a novel paradigm for efficient LLM knowledge coverage assessment without invoking the target LLM. Its core innovation lies in employing pretrained text embedding models—not graph embeddings—as lightweight surrogates, coupled with a linear decoder, to directly predict an LLM’s generation propensity toward factual statements. Evaluated on three Wikipedia-derived datasets across four mainstream LLMs and seven embedding models, PEEK achieves up to 90% accuracy. Experiments demonstrate that sentence embeddings significantly outperform graph embeddings; moreover, PEEK reduces probing overhead by orders of magnitude while precisely localizing LLM knowledge gaps. This enables efficient model diagnostics and facilitates targeted knowledge editing.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large language models (LLMs) acquire knowledge across diverse domains such as science, history, and geography encountered during generative pre-training. However, due to their stochasticity, it is difficult to predict what LLMs have acquired. Prior work has developed different ways to probe this knowledge by investigating the hidden representations, crafting specific task prompts, curating representative samples, and estimating their uncertainty. However, these methods require making forward passes through the underlying model to probe the LLM's knowledge about a specific fact, making them computationally expensive and time-consuming. To bridge this gap, we propose $ extbf{PEEK}$ or $ extbf{P}$roxy $ extbf{E}$mbeddings to $ extbf{E}$stimate $ extbf{K}$nowledge of LLMs, by leveraging the pre-trained embedding models that effectively encode factual knowledge as text or graphs as proxies for LLMs. First, we identify a training set of facts known by LLMs through various probing strategies and then adapt embedding models to predict the LLM outputs with a linear decoder layer. Comprehensive evaluation on $3$ Wikipedia-derived datasets, $4$ LLMs, and $7$ embedding models shows that embeddings can predict LLM knowledge on a held-out set with up to 90 % accuracy. Furthermore, we find that sentence embedding models are more suitable than graph embeddings to predict LLM knowledge, shedding light on the underlying representation of the factual landscape. Thus, we believe that knowledge-adapted embeddings can be used to identify knowledge gaps in LLMs at scale and can provide deeper insights into LLMs' internal inductive bias. The code and data are made available at https://github.com/claws-lab/peek.

Problem

• Probing LLM knowledge is computationally expensive
• Adapting embeddings predicts LLM knowledge efficiently
• Sentence embeddings outperform graph embeddings for prediction

Innovation

• Adapts pre-trained embeddings to predict LLM knowledge
• Uses linear decoder layer for efficient knowledge estimation
• Leverages sentence embeddings over graph embeddings