QuIM-RAG: Advancing Retrieval-Augmented Generation With Inverted Question Matching for Enhanced QA Performance

2025-01-06

IEEE Access

AI Summary

Retrieval-Augmented Generation (RAG) suffers from information dilution, frequent hallucinations, and insufficient semantic alignment between queries and retrieved documents. Method: We propose a “Question-to-Question” (Q2Q) inverted indexing paradigm: document chunks are reverse-engineered into semantically plausible questions, forming a question-level inverted index that enables fine-grained semantic matching between user queries and document content. Our system is built upon LLaMA3-8B-Instruct and trained on a corpus comprising over 500 pages of high-traffic web content. Contribution/Results: Evaluated rigorously under dual authoritative benchmarks—BERTScore and RAGAS—our approach significantly outperforms conventional RAG baselines. Results demonstrate superior accuracy, robustness, and generalization capability in complex, dynamic question-answering scenarios. The Q2Q paradigm introduces a novel, semantics-driven retrieval mechanism for RAG, shifting focus from lexical or embedding-based document retrieval to question-aligned, intent-aware matching.

Technology Category

• Natural Language Processing: Question Answering
• Data Mining & Knowledge Management: Intelligent Query Processing
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs

Abstract

This work presents a novel architecture for building Retrieval-Augmented Generation (RAG) systems to improve Question Answering (QA) tasks from a target corpus. Large Language Models (LLMs) have revolutionized the analyzing and generation of human-like text. These models rely on pre-trained data and lack real-time updates unless integrated with live data tools. RAG enhances LLMs by integrating online resources and databases to generate contextually appropriate responses. However, traditional RAG still encounters challenges like information dilution and hallucinations when handling vast amounts of data. Our approach addresses these challenges by converting corpora into a domain-specific dataset and RAG architecture is constructed to generate responses from the target document. We introduce QuIM-RAG (Question-to-question Inverted Index Matching), a novel approach for the retrieval mechanism in our system. This strategy generates potential questions from document chunks and matches these with user queries to identify the most relevant text chunks for generating accurate answers. We have implemented our RAG system on top of the open-source Meta-LLaMA3-8B-instruct model by Meta Inc. that is available on Hugging Face. We constructed a custom corpus of 500+ pages from a high-traffic website accessed thousands of times daily for answering complex questions, along with manually prepared ground truth QA for evaluation. We compared our approach with traditional RAG models using BERT-Score and RAGAS, state-of-the-art metrics for evaluating LLM applications. Our evaluation demonstrates that our approach outperforms traditional RAG architectures on both metrics.

Problem

• Information Accuracy
• Data Management
• Real-time Updating

Innovation

• QuIM-RAG
• Enhanced Retrieval Augmented Generation
• Accuracy Improvement