Privacy-protected Retrieval-Augmented Generation for Knowledge Graph Question Answering

2025-08-12

AI Summary

This work addresses privacy risks in knowledge graph question answering (KGQA) arising from retrieval-augmented generation (RAG), particularly semantic leakage of private knowledge graph entities when querying third-party black-box large language model (LLM) APIs. We propose ARoG, the first privacy-preserving RAG framework for KGQA. Its core innovation lies in a relation-centric abstraction mechanism and a structure-guided abstraction strategy: private entities are anonymized and mapped to high-level semantic concepts, while natural language questions are transformed into concept-based paths—enabling precise retrieval under semantic concealment. ARoG integrates dynamic relational semantic modeling, concept abstraction, and structured path conversion to achieve efficient retrieval and generation without exposing original entities. Experiments on three public benchmarks demonstrate that ARoG significantly improves retrieval accuracy under strong privacy guarantees, while maintaining superior performance and robustness.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Machine Learning: Privacy
• Natural Language Processing: Question Answering

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

LLMs often suffer from hallucinations and outdated or incomplete knowledge. RAG is proposed to address these issues by integrating external knowledge like that in KGs into LLMs. However, leveraging private KGs in RAG systems poses significant privacy risks due to the black-box nature of LLMs and potential insecure data transmission, especially when using third-party LLM APIs lacking transparency and control. In this paper, we investigate the privacy-protected RAG scenario for the first time, where entities in KGs are anonymous for LLMs, thus preventing them from accessing entity semantics. Due to the loss of semantics of entities, previous RAG systems cannot retrieve question-relevant knowledge from KGs by matching questions with the meaningless identifiers of anonymous entities. To realize an effective RAG system in this scenario, two key challenges must be addressed: (1) How can anonymous entities be converted into retrievable information. (2) How to retrieve question-relevant anonymous entities. Hence, we propose a novel ARoG framework including relation-centric abstraction and structure-oriented abstraction strategies. For challenge (1), the first strategy abstracts entities into high-level concepts by dynamically capturing the semantics of their adjacent relations. It supplements meaningful semantics which can further support the retrieval process. For challenge (2), the second strategy transforms unstructured natural language questions into structured abstract concept paths. These paths can be more effectively aligned with the abstracted concepts in KGs, thereby improving retrieval performance. To guide LLMs to effectively retrieve knowledge from KGs, the two strategies strictly protect privacy from being exposed to LLMs. Experiments on three datasets demonstrate that ARoG achieves strong performance and privacy-robustness.

Problem

• Address LLM hallucinations with privacy-protected RAG
• Retrieve knowledge from anonymous KG entities securely
• Convert anonymous entities into retrievable structured information

Innovation

• Relation-centric abstraction for anonymous entities
• Structure-oriented abstraction for question transformation
• Privacy-protected RAG framework ARoG