FDCT: Frequency-Aware Decomposition and Cross-Modal Token-Alignment for Multi-Sensor Target Classification

2025-03-12

IEEE Transactions on Aerospace and Electronic Systems

AI Summary

Feature fusion in multi-sensor automatic target recognition (ATR) is hindered by environmental interference, sensor heterogeneity, domain shift, granularity mismatch, and image misalignment. Method: This paper proposes a frequency-domain-aware decomposition and Unified Discrete Token (UDT) representation framework. It (i) introduces a novel frequency-domain decomposition module to disentangle detail and structural information; (ii) constructs a cross-modal UDT space enabling modality-agnostic semantic alignment; and (iii) designs a sparsity-constrained token-level alignment mechanism to jointly ensure discriminability and sensor robustness. Technically, it integrates frequency-domain modeling, decoupled representation learning (domain-specific/domain-invariant), sparse representation learning, and unified token embedding. Results: Evaluated on four multi-sensor ATR benchmarks, the method achieves state-of-the-art classification accuracy and interference robustness, significantly outperforming both unimodal baselines and mainstream multimodal fusion approaches.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

In automatic target recognition (ATR) systems, sensors may fail to capture discriminative, fine-grained detail features due to environmental conditions, noise created by CMOS chips, occlusion, parallaxes, and sensor misalignment. Therefore, multi-sensor image fusion is an effective choice to overcome these constraints. However, multi-modal image sensors are heterogeneous and have domain and granularity gaps. In addition, the multi-sensor images can be misaligned due to intricate background clutters, fluctuating illumination conditions, and uncontrolled sensor settings. In this paper, to overcome these issues, we decompose, align, and fuse multiple image sensor data for target classification. We extract the domain-specific and domain-invariant features from each sensor data. We propose to develop a shared unified discrete token (UDT) space between sensors to reduce the domain and granularity gaps. Additionally, we develop an alignment module to overcome the misalignment between multi-sensors and emphasize the discriminative representation of the UDT space. In the alignment module, we introduce sparsity constraints to provide a better cross-modal representation of the UDT space and robustness against various sensor settings. We achieve superior classification performance compared to single-modality classifiers and several state-of-the-art multi-modal fusion algorithms on four multi-sensor ATR datasets.

Problem

• Overcome sensor limitations in target recognition systems
• Address domain and granularity gaps in multi-sensor data
• Improve classification via cross-modal alignment and feature fusion

Innovation

• Decompose and align multi-sensor image data
• Create shared unified discrete token space
• Introduce sparsity constraints for robust alignment