Geometric Knowledge-Assisted Federated Dual Knowledge Distillation Approach Towards Remote Sensing Satellite Imagery

2026-03-08

AI Summary

This work addresses the performance degradation of federated learning in multi-satellite remote sensing imagery caused by data heterogeneity and distribution shifts. To this end, the authors propose a geometry-aware federated dual knowledge distillation framework. Each client employs a teacher–student network architecture that leverages globally aggregated geometric embeddings and covariance information from the server. Through a dual distillation mechanism and a multi-prototype generation strategy, the method enhances local feature representation. Crucially, geometric priors are innovatively incorporated to guide embedding learning, and a novel loss function is designed to align semantic structures across clients. Experimental results demonstrate significant improvements over state-of-the-art methods across multiple remote sensing benchmarks, achieving an average accuracy gain of 68.89% on EuroSAT using Swin-T as the backbone network.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Remote Sensing / Geospatial AI
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing

Abstract

Federated learning (FL) has recently become a promising solution for analyzing remote sensing satellite imagery (RSSI). However, the large scale and inherent data heterogeneity of images collected from multiple satellites, where the local data distribution of each satellite differs from the global one, present significant challenges to effective model training. To address this issue, we propose a Geometric Knowledge-Guided Federated Dual Knowledge Distillation (GK-FedDKD) framework for RSSI analysis. In our approach, each local client first distills a teacher encoder (TE) from multiple student encoders (SEs) trained with unlabeled augmented data. The TE is then connected with a shared classifier to form a teacher network (TN) that supervises the training of a new student network (SN). The intermediate representations of the TN are used to compute local covariance matrices, which are aggregated at the server to generate global geometric knowledge (GGK). This GGK is subsequently employed for local embedding augmentation to further guide SN training. We also design a novel loss function and a multi-prototype generation pipeline to stabilize the training process. Evaluation over multiple datasets showcases that the proposed GK-FedDKD approach is superior to the considered state-of-the-art baselines, e.g., the proposed approach with the Swin-T backbone surpasses previous SOTA approaches by an average 68.89% on the EuroSAT dataset.

Problem

• Federated Learning
• Remote Sensing Satellite Imagery
• Data Heterogeneity
• Model Training
• Geometric Knowledge

Innovation

• Federated Learning
• Knowledge Distillation
• Geometric Knowledge
• Remote Sensing
• Data Heterogeneity