JointCQ: Improving Factual Hallucination Detection with Joint Claim and Query Generation

2025-10-22

AI Summary

Large language models (LLMs) frequently generate factually inconsistent outputs—so-called hallucinations. Existing hallucination detection methods suffer from two critical limitations: (1) loss of contextual information during claim extraction, and (2) insufficient specificity in verification query generation. To address these issues, we propose the first end-to-end joint modeling framework that simultaneously optimizes claim extraction and verification query generation. Our approach employs fine-grained context-aware modeling to mitigate information decay and introduces a training strategy based on controllable synthetic data filtering to enhance query discriminability. Evaluated on multiple open-domain question-answering hallucination detection benchmarks—including HOVER and FEVER-Sym—our method achieves significant improvements over state-of-the-art approaches. Downstream retrieval-based verification accuracy increases by 4.2–7.8 percentage points on average, demonstrating enhanced robustness and interpretability in hallucination detection.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Current large language models (LLMs) often suffer from hallucination issues, i,e, generating content that appears factual but is actually unreliable. A typical hallucination detection pipeline involves response decomposition (i.e., claim extraction), query generation, evidence collection (i.e., search or retrieval), and claim verification. However, existing methods exhibit limitations in the first two stages, such as context loss during claim extraction and low specificity in query generation, resulting in degraded performance across the hallucination detection pipeline. In this work, we introduce JointCQ https://github.com/pku0xff/JointCQ, a joint claim-and-query generation framework designed to construct an effective and efficient claim-query generator. Our framework leverages elaborately designed evaluation criteria to filter synthesized training data, and finetunes a language model for joint claim extraction and query generation, providing reliable and informative inputs for downstream search and verification. Experimental results demonstrate that our method outperforms previous methods on multiple open-domain QA hallucination detection benchmarks, advancing the goal of more trustworthy and transparent language model systems.

Problem

• Detecting factual hallucinations in large language model outputs
• Improving claim extraction and query generation for verification
• Enhancing reliability and transparency of language model systems

Innovation

• Joint claim and query generation framework
• Filtered training data with evaluation criteria
• Finetuned model for improved hallucination detection