FreDFT: Frequency Domain Fusion Transformer for Visible-Infrared Object Detection

2025-11-13

AI Summary

To address cross-modal information imbalance and insufficient feature fusion in visible-infrared object detection, this paper proposes the Frequency-domain Fusion Transformer (FreDFT). It pioneers the integration of Transformer architectures into the frequency domain by introducing Frequency-domain Multimodal Attention (MFDA) and a Frequency-domain Feed-Forward Layer (FDFFL), jointly enhanced by Cross-modal Global Modeling (CGMM), Local Feature Enhancement (LFEM), and a hybrid-scale frequency-domain fusion strategy. Leveraging synergistic convolutional–frequency-domain modeling, FreDFT enables efficient cross-modal interaction and complementary feature mining. Extensive experiments on multiple public benchmarks demonstrate state-of-the-art performance, significantly outperforming existing spatial-domain methods. This work validates both the effectiveness and novelty of frequency-domain representation learning for multimodal object detection.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Visible-infrared object detection has gained sufficient attention due to its detection performance in low light, fog, and rain conditions. However, visible and infrared modalities captured by different sensors exist the information imbalance problem in complex scenarios, which can cause inadequate cross-modal fusion, resulting in degraded detection performance. extcolor{red}{Furthermore, most existing methods use transformers in the spatial domain to capture complementary features, ignoring the advantages of developing frequency domain transformers to mine complementary information.} To solve these weaknesses, we propose a frequency domain fusion transformer, called FreDFT, for visible-infrared object detection. The proposed approach employs a novel multimodal frequency domain attention (MFDA) to mine complementary information between modalities and a frequency domain feed-forward layer (FDFFL) via a mixed-scale frequency feature fusion strategy is designed to better enhance multimodal features. To eliminate the imbalance of multimodal information, a cross-modal global modeling module (CGMM) is constructed to perform pixel-wise inter-modal feature interaction in a spatial and channel manner. Moreover, a local feature enhancement module (LFEM) is developed to strengthen multimodal local feature representation and promote multimodal feature fusion by using various convolution layers and applying a channel shuffle. Extensive experimental results have verified that our proposed FreDFT achieves excellent performance on multiple public datasets compared with other state-of-the-art methods. The code of our FreDFT is linked at https://github.com/WenCongWu/FreDFT.

Problem

• Solving information imbalance between visible and infrared modalities in object detection
• Developing frequency domain transformers to mine complementary cross-modal information
• Addressing inadequate multimodal feature fusion in complex environmental conditions

Innovation

• Frequency domain attention mines cross-modal complementary information
• Mixed-scale frequency fusion enhances multimodal feature representation
• Cross-modal global modeling balances multimodal information interaction