FedMosaic: Federated Retrieval-Augmented Generation via Parametric Adapters

2026-02-05

AI Summary

This work addresses the limitations of conventional retrieval-augmented generation (RAG) in privacy-sensitive data silo scenarios, where reliance on centralized knowledge bases is infeasible, and existing federated RAG approaches suffer from high communication and storage overheads as well as adapter aggregation conflicts. To overcome these challenges, we propose FedMosaic—the first parameterized adapter-based federated RAG framework. FedMosaic constructs shared adapters across multiple documents via semantic clustering, augmented with document-specific masks and a selective aggregation mechanism, enabling efficient and accurate generation without transmitting raw documents. Experimental results demonstrate that FedMosaic improves average accuracy by 10.9% across four benchmark tasks while reducing storage costs by 78.8%–86.3% and cutting communication overhead by 91.4%.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by grounding generation in external knowledge to improve factuality and reduce hallucinations. Yet most deployments assume a centralized corpus, which is infeasible in privacy aware domains where knowledge remains siloed. This motivates federated RAG (FedRAG), where a central LLM server collaborates with distributed silos without sharing raw documents. In context RAG violates this requirement by transmitting verbatim documents, whereas parametric RAG encodes documents into lightweight adapters that merge with a frozen LLM at inference, avoiding raw-text exchange. We adopt the parametric approach but face two unique challenges induced by FedRAG: high storage and communication from per-document adapters, and destructive aggregation caused by indiscriminately merging multiple adapters. We present FedMosaic, the first federated RAG framework built on parametric adapters. FedMosaic clusters semantically related documents into multi-document adapters with document-specific masks to reduce overhead while preserving specificity, and performs selective adapter aggregation to combine only relevance-aligned, nonconflicting adapters. Experiments show that FedMosaic achieves an average 10.9% higher accuracy than state-of-the-art methods in four categories, while lowering storage costs by 78.8% to 86.3% and communication costs by 91.4%, and never sharing raw documents.

Problem

• Federated Retrieval-Augmented Generation
• privacy-preserving
• data silos
• parametric adapters
• federated learning

Innovation

• Federated Retrieval-Augmented Generation
• Parametric Adapters
• Selective Adapter Aggregation
• Document Clustering with Masks
• Privacy-Preserving LLM