URAG: Implementing a Unified Hybrid RAG for Precise Answers in University Admission Chatbots -- A Case Study at HCMUT

2025-01-27

AI Summary

To address the challenges of low answer accuracy, high risk of misinformation, and excessive deployment costs when applying large language models (LLMs) in university admissions counseling, this paper proposes a lightweight, training-free unified hybrid RAG framework. The method innovatively integrates semantic retrieval, keyword-augmented querying, multi-source result re-ranking, and LLM instruction-coordinated generation—requiring no fine-tuning and no specialized modules—thereby substantially reducing system complexity and deployment overhead. Deployed at Ho Chi Minh City University of Technology (HCMUT), our in-house lightweight model achieves accuracy on par with commercial LLMs for critical admissions Q&A tasks, while significantly lowering erroneous response rates. It has received strong endorsement from faculty and students. This work establishes a practical, cost-effective, and high-accuracy technical pathway for trustworthy educational question answering in resource-constrained environments.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Natural Language Processing: Question Answering

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

With the rapid advancement of Artificial Intelligence, particularly in Natural Language Processing, Large Language Models (LLMs) have become pivotal in educational question-answering systems, especially university admission chatbots. Concepts such as Retrieval-Augmented Generation (RAG) and other advanced techniques have been developed to enhance these systems by integrating specific university data, enabling LLMs to provide informed responses on admissions and academic counseling. However, these enhanced RAG techniques often involve high operational costs and require the training of complex, specialized modules, which poses challenges for practical deployment. Additionally, in the educational context, it is crucial to provide accurate answers to prevent misinformation, a task that LLM-based systems find challenging without appropriate strategies and methods. In this paper, we introduce the Unified RAG (URAG) Framework, a hybrid approach that significantly improves the accuracy of responses, particularly for critical queries. Experimental results demonstrate that URAG enhances our in-house, lightweight model to perform comparably to state-of-the-art commercial models. Moreover, to validate its practical applicability, we conducted a case study at our educational institution, which received positive feedback and acclaim. This study not only proves the effectiveness of URAG but also highlights its feasibility for real-world implementation in educational settings.

Problem

• University Admissions
• Chatbot Accuracy
• User Misleading Prevention

Innovation

• Unified Hybrid Method
• Retrieval-Augmented Generation Optimization
• Education Chatbot Accuracy Enhancement