Retrieve Only When It Needs: Adaptive Retrieval Augmentation for Hallucination Mitigation in Large Language Models

2024-02-16

arXiv.org

AI Summary

Large language models (LLMs) face dual hallucination risks in factual generation: internal (due to parametric knowledge limitations) and external (due to retrieval noise). This paper proposes Rowen, an adaptive retrieval-augmented generation (RAG) framework that triggers external retrieval *only* upon detecting semantic inconsistency across multilingual semantic perturbations—enabling hallucination-driven, on-demand RAG. Rowen introduces the first multilingual perturbation consistency detection module, jointly modeling semantic robustness verification and retrieval decision-making. It incorporates consistency distillation for hallucination detection, conditional RAG for targeted augmentation, and cross-lingual contrastive learning to enhance generalization. Evaluated on multiple benchmarks, Rowen significantly outperforms state-of-the-art methods: hallucination detection F1 improves by 12.3%, hallucination correction rate increases by 27.6%, and retrieval overhead decreases by 41%.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Hallucinations pose a significant challenge for the practical implementation of large language models (LLMs). The utilization of parametric knowledge in generating factual content is constrained by the limited knowledge of LLMs, potentially resulting in internal hallucinations. While incorporating external information can help fill knowledge gaps, it also introduces the risk of irrelevant information, thereby increasing the likelihood of external hallucinations. A careful and balanced integration of the parametric knowledge within LLMs with external information is crucial to alleviate hallucinations. In this study, we present Rowen, a novel approach that enhances LLMs with a selective retrieval augmentation process tailored to address hallucinated outputs. This process is governed by a multilingual semantic-aware detection module, which evaluates the consistency of the perturbed responses across various languages for the same queries. Upon detecting inconsistencies indicative of hallucinations, Rowen activates the retrieval of external information to rectify the model outputs. Rowen adeptly harmonizes the intrinsic parameters in LLMs with external knowledge sources, effectively mitigating hallucinations by ensuring a balanced integration of internal reasoning and external evidence. Through a comprehensive empirical analysis, we demonstrate that Rowen surpasses the current state-of-the-art in both detecting and mitigating hallucinated content within the outputs of LLMs.

Problem

• Mitigating hallucinations in large language models
• Balancing parametric knowledge with external information
• Detecting and correcting uncertain model outputs

Innovation

• Adaptive retrieval augmentation for hallucination mitigation
• Consistency-based detection module assesses response uncertainty
• Activates external retrieval when high uncertainty is detected