Supercharging Federated Intelligence Retrieval

2026-03-26

AI Summary

This work addresses the challenge of applying traditional Retrieval-Augmented Generation (RAG) in scenarios where relevant knowledge is fragmented across multiple private data silos. We propose the first RAG system that integrates confidential computing with federated retrieval: each participant performs local retrieval, while a central server—operating within a remotely attested Trusted Execution Environment (TEE)—securely aggregates the results and generates the final response. To enhance generation quality without compromising privacy, we introduce a novel cascaded inference mechanism that safely incorporates non-confidential third-party large language models. Implemented on the Flower framework, our system ensures end-to-end confidentiality against honest-but-curious or compromised servers while significantly improving the accuracy and practical utility of generated answers.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Privacy
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Security and Privacy: Data transparency and provenance

Abstract

RAG typically assumes centralized access to documents, which breaks down when knowledge is distributed across private data silos. We propose a secure Federated RAG system built using Flower that performs local silo retrieval, while server-side aggregation and text generation run inside an attested, confidential compute environment, enabling confidential remote LLM inference even in the presence of honest-but-curious or compromised servers. We also propose a cascading inference approach that incorporates a non-confidential third-party model (e.g., Amazon Nova) as auxiliary context without weakening confidentiality.

Problem

• Federated RAG
• confidential computing
• private data silos
• secure retrieval
• LLM inference

Innovation

• Federated RAG
• Confidential Computing
• Secure Aggregation
• Cascading Inference
• Private Data Silos