Enhancing Document-Level Question Answering via Multi-Hop Retrieval-Augmented Generation with LLaMA 3

2025-06-19

AI Summary

To address challenges in long-document multi-hop question answering—including difficulty in cross-paragraph reasoning, weak contextual understanding, and answer inconsistency—this paper proposes a retrieval-augmented generation (RAG) framework based on LLaMA-3. Methodologically: (1) it introduces a joint optimization objective that simultaneously minimizes retrieval likelihood loss and generation cross-entropy loss; (2) it designs a hierarchical multi-hop retrieval–generation coordination mechanism integrating a DPR variant, a multi-hop graph reasoning module, and a context-aware fusion encoder; and (3) it employs end-to-end joint training. The framework achieves state-of-the-art performance on HotpotQA, MuSiQue, and DocQA—improving multi-hop accuracy by 7.2% and answer faithfulness by 11.5% over baseline RAG and pure generative methods. Its core contribution lies in the first unified modeling of retrieval and generation objectives, coupled with deep synergy between multi-hop logical reasoning and contextual representation learning.

Technology Category

• Natural Language Processing: Question Answering
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

This paper presents a novel Retrieval-Augmented Generation (RAG) framework tailored for complex question answering tasks, addressing challenges in multi-hop reasoning and contextual understanding across lengthy documents. Built upon LLaMA 3, the framework integrates a dense retrieval module with advanced context fusion and multi-hop reasoning mechanisms, enabling more accurate and coherent response generation. A joint optimization strategy combining retrieval likelihood and generation cross-entropy improves the model's robustness and adaptability. Experimental results show that the proposed system outperforms existing retrieval-augmented and generative baselines, confirming its effectiveness in delivering precise, contextually grounded answers.

Problem

• Enhancing document-level QA via multi-hop RAG
• Improving multi-hop reasoning in lengthy documents
• Optimizing retrieval-generation synergy for accurate answers

Innovation

• Multi-hop RAG framework with LLaMA 3
• Dense retrieval and context fusion
• Joint optimization of retrieval-generation