EGM-Det: Entropy-Guided Multimodal Adaptive Fusion for UAV RGB-IR Object Detection

2026-08-12

AI Summary

This work addresses the limitations of existing RGB-IR object detection methods for unmanned aerial vehicles, which typically rely on static fusion strategies that fail to account for spatially varying modality reliability. To overcome this, the authors propose EGM-Det, a dual-stream framework that preserves modality-specific representations and introduces an entropy-guided offset gating mechanism. This mechanism leverages input intensity, local entropy, and cross-modal discrepancies to construct shallow entropy priors, dynamically guiding multi-scale spatial-channel alignment and fusion. Additionally, an entropy-adaptive supervised cross-modal knowledge distillation strategy is designed to optimize training. The proposed method achieves state-of-the-art performance across three benchmarks—DroneVehicle, LLVIP, and VEDAI—with a notable improvement of over 10 percentage points in mAP on VEDAI.

Abstract

Joint use of RGB and infrared (IR) imagery can improve UAV-view object detection, but most existing methods fuse multimodal features with static or fixed weights and therefore overlook spatially varying modality reliability. We propose EGM-Det, an entropy-guided multimodal adaptive fusion framework for RGB-IR object detection. EGM-Det employs a dual-stream architecture to preserve modality-specific representations and introduces an Entropy Offset Gate Fusion module for adaptive multi-scale fusion. The module derives shallow entropy priors from input intensity, local entropy, and cross-modal discrepancy, and uses them to guide local offset alignment and spatial-channel gated fusion. It therefore selectively aggregates reliable RGB and infrared cues instead of uniformly combining heterogeneous features. We further introduce cross-modal distillation to regularize the learned fusion gates and reduce fusion degradation. Each student branch extracts complementary knowledge from the cross-modality teacher branch matched to the main branch, while entropy-adaptive supervision emphasizes uncertain modality decisions. Experiments on DroneVehicle, LLVIP, and VEDAI demonstrate state-of-the-art performance across all three benchmarks; in particular, EGM-Det outperforms prior approaches by more than 10 percentage points on VEDAI.

Problem

• multimodal fusion
• UAV object detection
• modality reliability
• RGB-IR imagery
• adaptive fusion

Innovation

• entropy-guided fusion
• adaptive multimodal fusion
• RGB-IR object detection
• cross-modal distillation
• UAV perception