Towards Robust Optical-SAR Object Detection under Missing Modalities: A Dynamic Quality-Aware Fusion Framework

2025-12-26

AI Summary

To address the insufficient robustness in optical and SAR remote sensing image fusion detection caused by modality missing, misalignment, and quality degradation, this paper proposes a dynamic quality-aware fusion framework. Methodologically, it introduces (1) a learnable reference-token-driven Dynamic Modality Quality Assessment (DMQA) for fine-grained reliability estimation, and (2) an Orthogonal-Constrained Normalized Fusion (OCNF) module to ensure feature decoupling and adaptive weighting. The framework maintains stable detection performance under stochastic modality absence. Evaluated on SpaceNet6-OTD and OGSOD-2.0 benchmarks, it significantly outperforms state-of-the-art methods: under 50% modality missing, it achieves an 8.7% mAP improvement. The approach demonstrates strong robustness, generalizability, and operational reliability for all-weather object detection.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Optical and Synthetic Aperture Radar (SAR) fusion-based object detection has attracted significant research interest in remote sensing, as these modalities provide complementary information for all-weather monitoring. However, practical deployment is severely limited by inherent challenges. Due to distinct imaging mechanisms, temporal asynchrony, and registration difficulties, obtaining well-aligned optical-SAR image pairs remains extremely difficult, frequently resulting in missing or degraded modality data. Although recent approaches have attempted to address this issue, they still suffer from limited robustness to random missing modalities and lack effective mechanisms to ensure consistent performance improvement in fusion-based detection. To address these limitations, we propose a novel Quality-Aware Dynamic Fusion Network (QDFNet) for robust optical-SAR object detection. Our proposed method leverages learnable reference tokens to dynamically assess feature reliability and guide adaptive fusion in the presence of missing modalities. In particular, we design a Dynamic Modality Quality Assessment (DMQA) module that employs learnable reference tokens to iteratively refine feature reliability assessment, enabling precise identification of degraded regions and providing quality guidance for subsequent fusion. Moreover, we develop an Orthogonal Constraint Normalization Fusion (OCNF) module that employs orthogonal constraints to preserve modality independence while dynamically adjusting fusion weights based on reliability scores, effectively suppressing unreliable feature propagation. Extensive experiments on the SpaceNet6-OTD and OGSOD-2.0 datasets demonstrate the superiority and effectiveness of QDFNet compared to state-of-the-art methods, particularly under partial modality corruption or missing data scenarios.

Problem

• Robust object detection with missing optical-SAR modalities
• Dynamic quality-aware fusion for degraded multimodal data
• Preserving modality independence while suppressing unreliable features

Innovation

• Dynamic quality-aware fusion with learnable reference tokens
• Orthogonal constraints preserve modality independence during fusion
• Adaptive weight adjustment based on reliability scores for robustness