SINdex: Semantic INconsistency Index for Hallucination Detection in LLMs

2025-03-07

AI Summary

To address the pervasive factual hallucination problem in large language model (LLM) outputs, this paper proposes an uncertainty-driven semantic clustering framework for hallucination detection—requiring no external supervision and compatible with any standard LLM. The method introduces SINdex, the first unsupervised semantic inconsistency metric, which quantifies internal semantic contradictions among model-generated answers via SBERT sentence embeddings and hierarchical clustering, augmented by uncertainty calibration to enhance discriminative robustness. Crucially, the approach operates entirely without external knowledge bases or annotated data. Evaluated on both open- and closed-book QA benchmarks, it achieves up to a 9.3% improvement in AUROC over state-of-the-art hallucination detection methods. The framework demonstrates strong generalization across diverse LLMs and tasks, and offers plug-and-play practicality for real-world deployment.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Reasoning under Uncertainty: Uncertainty Representations

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Large language models (LLMs) are increasingly deployed across diverse domains, yet they are prone to generating factually incorrect outputs - commonly known as"hallucinations."Among existing mitigation strategies, uncertainty-based methods are particularly attractive due to their ease of implementation, independence from external data, and compatibility with standard LLMs. In this work, we introduce a novel and scalable uncertainty-based semantic clustering framework for automated hallucination detection. Our approach leverages sentence embeddings and hierarchical clustering alongside a newly proposed inconsistency measure, SINdex, to yield more homogeneous clusters and more accurate detection of hallucination phenomena across various LLMs. Evaluations on prominent open- and closed-book QA datasets demonstrate that our method achieves AUROC improvements of up to 9.3% over state-of-the-art techniques. Extensive ablation studies further validate the effectiveness of each component in our framework.

Problem

• Detects hallucinations in large language models (LLMs).
• Uses semantic clustering for automated inconsistency detection.
• Improves AUROC by 9.3% over state-of-the-art methods.

Innovation

• Uncertainty-based semantic clustering for hallucination detection
• Uses SINdex for inconsistency measurement in LLMs
• Improves AUROC by up to 9.3% over existing methods