A novel hallucination classification framework

2025-10-06

AI Summary

This paper addresses the challenge of hallucination detection in large language model (LLM) inference. Methodologically, it proposes a lightweight, unsupervised automatic detection framework: first, a systematic hallucination taxonomy is constructed, and diverse hallucinated responses are controllably generated via prompt engineering; second, response embeddings—obtained using a pre-trained embedding model—are projected into a low-dimensional space, where dimensionality reduction and unsupervised clustering reveal distributional separability between hallucinated and truthful responses; finally, the distance from each response’s embedding to the centroid of truthful responses quantifies its hallucination severity. Key contributions include: (i) the first demonstration of controllable hallucination re-generation via prompt engineering; (ii) empirical discovery of a strong correlation between hallucination severity and embedding-space distance to the truthful centroid; and (iii) validation that even simple classifiers achieve high detection accuracy, confirming the framework’s effectiveness in enhancing LLM reliability and its practical deployability.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

This work introduces a novel methodology for the automatic detection of hallucinations generated during large language model (LLM) inference. The proposed approach is based on a systematic taxonomy and controlled reproduction of diverse hallucination types through prompt engineering. A dedicated hallucination dataset is subsequently mapped into a vector space using an embedding model and analyzed with unsupervised learning techniques in a reduced-dimensional representation of hallucinations with veridical responses. Quantitative evaluation of inter-centroid distances reveals a consistent correlation between the severity of informational distortion in hallucinations and their spatial divergence from the cluster of correct outputs. These findings provide theoretical and empirical evidence that even simple classification algorithms can reliably distinguish hallucinations from accurate responses within a single LLM, thereby offering a lightweight yet effective framework for improving model reliability.

Problem

• Automatically detecting hallucinations in large language model outputs
• Classifying hallucinations by severity using spatial divergence analysis
• Providing lightweight framework to distinguish hallucinations from accurate responses

Innovation

• Novel hallucination classification framework using taxonomy
• Unsupervised learning in vector space for detection
• Lightweight classification algorithms distinguish hallucinations reliably