On the Universal Truthfulness Hyperplane Inside LLMs

2024-07-11

Conference on Empirical Methods in Natural Language Processing

AI Summary

Large language models (LLMs) frequently generate factually incorrect responses to novel inputs, revealing an intrinsic lack of factual discrimination capability. This work systematically investigates, for the first time, whether a generalizable “truth hyperplane”—a linear decision boundary in hidden-layer representations that separates factually correct from incorrect outputs—exists within LLMs. We jointly train a linear classifier on hidden-state representations across 40+ diverse, heterogeneous datasets and evaluate its robustness via cross-task and cross-domain zero-shot transfer. Results demonstrate that training data diversity is markedly more critical than scale for learning stable, generalizable truth-discriminative boundaries; such hyperplanes are pervasive across models and tasks, providing the first empirical evidence that LLMs possess an intrinsic, representation-level mechanism for factual awareness.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Knowledge Representation and Reasoning: Reasoning with Beliefs

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

While large language models (LLMs) have demonstrated remarkable abilities across various fields, hallucination remains a significant challenge. Recent studies have explored hallucinations through the lens of internal representations, proposing mechanisms to decipher LLMs’ adherence to facts. However, these approaches often fail to generalize to out-of-distribution data, leading to concerns about whether internal representation patterns reflect fundamental factual awareness, or only overfit spurious correlations on the specific datasets. In this work, we investigate whether a universal truthfulness hyperplane that distinguishes the model’s factually correct and incorrect outputs exists within the model. To this end, we scale up the number of training datasets and conduct an extensive evaluation – we train the truthfulness hyperplane on a diverse collection of over 40 datasets and examine its cross-task, cross-domain, and in-domain generalization. Our results indicate that increasing the diversity of the training datasets significantly enhances the performance in all scenarios, while the volume of data samples plays a less critical role. This finding supports the optimistic hypothesis that a universal truthfulness hyperplane may indeed exist within the model, offering promising directions for future research.

Problem

• Large Language Models
• Factuality
• Novel Situations

Innovation

• Fact Judgement Mechanism
• Large Language Models
• Credibility Enhancement