A Factuality and Diversity Reconciled Decoding Method for Knowledge-Grounded Dialogue Generation

2024-07-08

arXiv.org

AI Summary

In knowledge-grounded dialogue, balancing factual consistency and response diversity remains challenging: over-reliance on external knowledge often leads to repetitive outputs, while stochastic decoding undermines factual accuracy. This paper proposes DoGe, a novel confidence-aware decoding framework that dynamically orchestrates internal parametric knowledge and external retrieved knowledge based on the model’s real-time factual confidence in the current generation segment—without fine-tuning or additional training. DoGe integrates internal knowledge modeling, external knowledge retrieval, and confidence-sensitive gating into a single controllable decoding process. Evaluated on three benchmark datasets, DoGe significantly outperforms existing decoding baselines: it improves factual accuracy (FActScore) by 8.7% and lexical diversity (Dist-2) by 12.3%, thereby achieving a principled unification of factual reliability and generative diversity.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Grounding external knowledge can enhance the factuality of responses in dialogue generation. However, excessive emphasis on it might result in the lack of engaging and diverse expressions. Through the introduction of randomness in sampling, current approaches can increase the diversity. Nevertheless, such sampling method could undermine the factuality in dialogue generation. In this study, to discover a solution for advancing creativity without relying on questionable randomness and to subtly reconcile the factuality and diversity within the source-grounded paradigm, a novel method named DoGe is proposed. DoGe can dynamically alternate between the utilization of internal parameter knowledge and external source knowledge based on the model's factual confidence. Extensive experiments on three widely-used datasets show that DoGe can not only enhance response diversity but also maintain factuality, and it significantly surpasses other various decoding strategy baselines.

Problem

• Reconciling factuality and diversity in dialogue generation
• Reducing reliance on random sampling for creativity
• Balancing internal and external knowledge utilization dynamically

Innovation

• Dynamic alternation between internal and external knowledge
• Factual confidence-based knowledge utilization
• Enhancing diversity while maintaining factuality