Prompt Embedding Probes (PEP): Hallucination Detection in LLMs from Hidden States

2026-08-08

AI Summary

Large language models (LLMs) are prone to hallucinations, necessitating efficient detection methods. This work proposes Prompt Embedding Probes (PEP), a white-box framework for hallucination detection that enhances the capability of linear probes operating on hidden states by injecting a small set of learnable prompt embeddings into the input, while keeping the LLM backbone frozen. Requiring only minimal additional parameters, PEP substantially improves answer-level hallucination identification and supports both pre-generated predictions and cross-model transfer. Experimental results demonstrate that PEP outperforms standard linear probes on TriviaQA, GSM8K, and MedQA benchmarks; however, challenges remain in generalizing across datasets.

Abstract

Large language models (LLMs) can generate fluent and useful responses but remain prone to hallucinations. We introduce Prompt Embedding Probes (PEP), a white-box method for answer-level hallucination detection from the hidden states of a frozen LLM. PEP extends standard linear probes by augmenting the input with a small number of learnable prompt embeddings. We evaluate PEP on TriviaQA, GSM8K, and MedQA using Qwen3 models at multiple scales. PEP improves hidden-state-based detection over standard linear probes in the main in-distribution setting. We further evaluate PEP for pre-generation prediction, cross-model transfer, and out-of-distribution generalization. PEP remains effective in the pre-generation and cross-model settings, whereas robust cross-dataset transfer remains difficult. These results show that prompt-based adaptation can strengthen hidden-state probing while keeping the backbone frozen and adding only a small number of trainable parameters.

Problem

• hallucination detection
• large language models
• hidden states
• prompt embedding
• white-box probing

Innovation

• Prompt Embedding Probes
• hallucination detection
• hidden states
• frozen LLM
• linear probing