FedEU: Evidential Uncertainty-Driven Federated Fine-Tuning of Vision Foundation Models for Remote Sensing Image Segmentation

2026-03-08

AI Summary

This work addresses the high uncertainty in model updates and unreliable collaborative optimization in federated learning for remote sensing image segmentation, primarily caused by client data heterogeneity. To tackle this challenge, the authors propose a federated optimization framework grounded in evidential uncertainty. The approach integrates parameter-efficient fine-tuning (PEFT), client-specific feature embeddings (CFE), and a personalized attention mechanism to model local uncertainty via evidential deep learning. Furthermore, a Top-k uncertainty-guided global aggregation strategy is devised to enable efficient and robust federated fine-tuning of the base segmentation model. Experiments on three large-scale heterogeneous remote sensing datasets demonstrate that the proposed method significantly reduces prediction uncertainty and enhances cross-client generalization and model reliability.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Stochastic Optimization
• Computer Vision: Segmentation

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Remote sensing image segmentation (RSIS) in federated environments has gained increasing attention because it enables collaborative model training across distributed datasets without sharing raw imagery or annotations. Federated RSIS combined with parameter-efficient fine-tuning (PEFT) can unleash the generalization power of pretrained foundation models for real-world applications, with minimal parameter aggregation and communication overhead. However, the dynamic adaptation of pretrained models to heterogeneous client data inevitably increases update uncertainty and compromises the reliability of collaborative optimization due to the lack of uncertainty estimation for each local model. To bridge this gap, we present FedEU, a federated optimization framework for fine-tuning RSIS models driven by evidential uncertainty. Specifically, personalized evidential uncertainty modeling is introduced to quantify epistemic variations of local models and identify high-risk areas under local data distributions. Furthermore, the client-specific feature embedding (CFE) is exploited to enhance channel-aware feature representation while preserving client-specific properties through personalized attention and an element-aware parameter update approach. These uncertainty estimates are uploaded to the server to enable adaptive global aggregation via a Top-k uncertainty-guided weighting (TUW) strategy, which mitigates the impact of distribution shifts and unreliable updates. Extensive experiments on three large-scale heterogeneous datasets demonstrate the superior performance of FedEU. More importantly, FedEU enables balanced model adaptation across diverse clients by explicitly reducing prediction uncertainty, resulting in more robust and reliable federated outcomes. The source codes will be available at https://github.com/zxk688/FedEU.

Problem

• federated learning
• remote sensing image segmentation
• evidential uncertainty
• heterogeneous data
• model reliability

Innovation

• evidential uncertainty
• federated fine-tuning
• personalized feature embedding
• uncertainty-guided aggregation
• vision foundation models