Learning Auxiliary Tasks Improves Reference-Free Hallucination Detection in Open-Domain Long-Form Generation

2025-05-18

AI Summary

To address the limitations of existing hallucination detection methods for open-domain long-text generation—namely, their reliance on external tools and poor generalizability—this paper proposes a lightweight, reference-free, internal-state-aware approach that requires no external knowledge retrieval. The core innovation is the RATE-FT paradigm: an instruction-tuning framework that jointly optimizes the primary task (paragraph-level hallucination classification) and an auxiliary task (token-level confidence prediction), while systematically modeling internal model states—including output probabilities and entropy. RATE-FT is architecture-agnostic and compatible with diverse mainstream LLMs. On the LongFact benchmark, it achieves a 3% absolute improvement in detection accuracy over standard fine-tuning baselines, significantly outperforming prompt engineering and probing-based methods. Moreover, it demonstrates strong cross-model generalization capability.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Hallucination, the generation of factually incorrect information, remains a significant challenge for large language models (LLMs), especially in open-domain long-form generation. Existing approaches for detecting hallucination in long-form tasks either focus on limited domains or rely heavily on external fact-checking tools, which may not always be available. In this work, we systematically investigate reference-free hallucination detection in open-domain long-form responses. Our findings reveal that internal states (e.g., model's output probability and entropy) alone are insufficient for reliably (i.e., better than random guessing) distinguishing between factual and hallucinated content. To enhance detection, we explore various existing approaches, including prompting-based methods, probing, and fine-tuning, with fine-tuning proving the most effective. To further improve the accuracy, we introduce a new paradigm, named RATE-FT, that augments fine-tuning with an auxiliary task for the model to jointly learn with the main task of hallucination detection. With extensive experiments and analysis using a variety of model families&datasets, we demonstrate the effectiveness and generalizability of our method, e.g., +3% over general fine-tuning methods on LongFact.

Problem

• Detecting hallucination in open-domain long-form generation without references
• Overcoming limitations of internal states for reliable hallucination detection
• Improving accuracy with auxiliary task-augmented fine-tuning (RATE-FT)

Innovation

• Uses fine-tuning for hallucination detection
• Introduces RATE-FT with auxiliary task
• Improves accuracy by 3% on LongFact