Alleviating LLM-based Generative Retrieval Hallucination in Alipay Search

2025-03-27

AI Summary

This work addresses the problem of hallucinated, query-irrelevant documents generated by large language models (LLMs) in generative retrieval (GR), which severely undermines result reliability. To tackle this, we propose an optimization framework integrating knowledge distillation and a multi-perspective decision agent. Specifically, we leverage LLMs for query–document (q-d) relevance reasoning and distill their discriminative capability into a lightweight GR model. We further design a decision agent that dynamically post-processes and refines retrieval results from multiple sources. Offline experiments on the Alipay search platform demonstrate a significant reduction in hallucination rates. Online A/B tests show measurable improvements in retrieval quality and user conversion rates for fund and insurance queries. To our knowledge, this is the first work to jointly introduce LLM-based reasoning distillation and decision-agent-based refinement into GR systems, effectively balancing generation fidelity and interpretability.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Generative retrieval (GR) has revolutionized document retrieval with the advent of large language models (LLMs), and LLM-based GR is gradually being adopted by the industry. Despite its remarkable advantages and potential, LLM-based GR suffers from hallucination and generates documents that are irrelevant to the query in some instances, severely challenging its credibility in practical applications. We thereby propose an optimized GR framework designed to alleviate retrieval hallucination, which integrates knowledge distillation reasoning in model training and incorporate decision agent to further improve retrieval precision. Specifically, we employ LLMs to assess and reason GR retrieved query-document (q-d) pairs, and then distill the reasoning data as transferred knowledge to the GR model. Moreover, we utilize a decision agent as post-processing to extend the GR retrieved documents through retrieval model and select the most relevant ones from multi perspectives as the final generative retrieval result. Extensive offline experiments on real-world datasets and online A/B tests on Fund Search and Insurance Search in Alipay demonstrate our framework's superiority and effectiveness in improving search quality and conversion gains.

Problem

• Reducing hallucination in LLM-based generative retrieval
• Improving relevance of retrieved documents to queries
• Enhancing search quality and conversion gains

Innovation

• Knowledge distillation reasoning in training
• Decision agent for post-processing refinement
• Multi-perspective document selection