Towards Privacy-preserved Pre-training of Remote Sensing Foundation Models with Federated Mutual-guidance Learning

2025-03-14

AI Summary

To address the dual challenges of data silos and privacy leakage in remote sensing foundation model (RSFM) pretraining, this paper proposes FedSense—a federated collaborative pretraining framework. Its core innovation is the first-ever federated mutual-guidance learning paradigm, establishing bidirectional guidance mechanisms: server-to-client guidance (SCG) and client-to-server guidance (CSG). FedSense synergistically integrates federated learning, self-supervised pretraining, model flatness optimization, low-bit knowledge distillation, and multi-source heterogeneous remote sensing modeling. Crucially, it eliminates the need for raw data sharing, thereby mitigating model drift caused by data heterogeneity and substantially reducing communication overhead. Evaluated on four downstream remote sensing tasks, FedSense achieves comparable or improved accuracy while cutting total communication volume by 60%. The framework significantly enhances training efficiency, privacy preservation, and cross-domain generalization capability.

Technology Category

• Computer Vision: Remote Sensing / Geospatial AI
• Machine Learning: Distributed Machine Learning & Federated Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing

Abstract

Traditional Remote Sensing Foundation models (RSFMs) are pre-trained with a data-centralized paradigm, through self-supervision on large-scale curated remote sensing data. For each institution, however, pre-training RSFMs with limited data in a standalone manner may lead to suboptimal performance, while aggregating remote sensing data from multiple institutions for centralized pre-training raises privacy concerns. Seeking for collaboration is a promising solution to resolve this dilemma, where multiple institutions can collaboratively train RSFMs without sharing private data. In this paper, we propose a novel privacy-preserved pre-training framework (FedSense), which enables multiple institutions to collaboratively train RSFMs without sharing private data. However, it is a non-trivial task hindered by a vicious cycle, which results from model drift by remote sensing data heterogeneity and high communication overhead. To break this vicious cycle, we introduce Federated Mutual-guidance Learning. Specifically, we propose a Server-to-Clients Guidance (SCG) mechanism to guide clients updates towards global-flatness optimal solutions. Additionally, we propose a Clients-to-Server Guidance (CSG) mechanism to inject local knowledge into the server by low-bit communication. Extensive experiments on four downstream tasks demonstrate the effectiveness of our FedSense in both full-precision and communication-reduced scenarios, showcasing remarkable communication efficiency and performance gains.

Problem

• Privacy concerns in centralized pre-training of remote sensing models.
• Suboptimal performance from standalone pre-training with limited data.
• High communication overhead and model drift in collaborative training.

Innovation

• Federated Mutual-guidance Learning for privacy
• Server-to-Clients Guidance mechanism
• Clients-to-Server Guidance with low-bit communication