KG2QA: Knowledge Graph-enhanced Retrieval-Augmented Generation for Communication Standards Question Answering

2025-06-08

AI Summary

To address the heavy reliance on domain experts and prolonged response times in telecommunications standards consultation, this paper proposes a knowledge graph-enhanced Retrieval-Augmented Generation (RAG) framework. Methodologically, it introduces a novel synergistic architecture integrating an ontology-driven knowledge graph (13,906 entities / 13,524 relations) with a LoRA-finetuned Qwen2.5-7B-Instruct large language model, enabling graph-guided precise retrieval and joint reasoning for domain-specific question answering. A DeepSeek-Judge–based automated evaluation mechanism is further incorporated. Experimental results demonstrate substantial improvements: BLEU-4 reaches 66.8993 (+48.04 absolute gain), ROUGE scores increase significantly, and five-dimensional human evaluation yields a mean score improvement of 2.26%. The system supports both API and web-based deployment, establishing a practical, deployable intelligent consultation paradigm for standards development.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Natural Language Processing: Question Answering
• Knowledge Representation and Reasoning: Knowledge Acquisition

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

There are many types of standards in the field of communication. The traditional consulting model has a long cycle and relies on the knowledge and experience of experts, making it difficult to meet the rapidly developing technological demands. This paper combines the fine-tuning of large language models with the construction of knowledge graphs to implement an intelligent consultation and question-answering system for communication standards. The experimental results show that after LoRA tuning on the constructed dataset of 6,587 questions and answers in the field of communication standards, Qwen2.5-7B-Instruct demonstrates outstanding professional capabilities in the field of communication standards on the test set. BLEU-4 rose from 18.8564 to 66.8993, and evaluation indicators such as ROUGE also increased significantly, outperforming the fine-tuning effect of the comparison model Llama-3-8B-Instruct. Based on the ontology framework containing 6 entity attributes and 10 relation attributes, a knowledge graph of the communication standard domain containing 13,906 entities and 13,524 relations was constructed, showing a relatively good query accuracy rate. The intelligent consultation and question-answering system enables the fine-tuned model on the server side to access the locally constructed knowledge graph and conduct graphical retrieval of key information first, which is conducive to improving the question-answering effect. The evaluation using DeepSeek as the Judge on the test set shows that our RAG framework enables the fine-tuned model to improve the scores at all five angles, with an average score increase of 2.26%. And combined with web services and API interfaces, it has achieved very good results in terms of interaction experience and back-end access, and has very good practical application value.

Problem

• Enhances QA for communication standards using KG and RAG
• Improves expert-dependent, slow traditional consultation methods
• Boosts model performance via fine-tuning and knowledge graphs

Innovation

• Fine-tuned Qwen2.5-7B-Instruct with LoRA
• Built domain-specific knowledge graph
• Integrated retrieval-augmented generation framework