Dynamic Cross-Modal Feature Interaction Network for Hyperspectral and LiDAR Data Classification

2025-03-10

AI Summary

To address the strong reliance on handcrafted features and insufficient cross-modal interaction in hyperspectral image (HSI) and LiDAR data fusion for classification, this paper proposes the first cross-modal feature interaction network based on a dynamic routing mechanism. Methodologically, we introduce a dynamic routing space and a routing gating mechanism, and design bilinear spatial/channel attention blocks (BSAB/BCAB) alongside an integrated convolutional block (ICB) to enable adaptive协同 enhancement and fusion of spatial, spectral, and discriminative features. Extensive experiments on three benchmark HSI–LiDAR datasets demonstrate that our approach consistently outperforms state-of-the-art methods by significant margins, achieving superior accuracy and robustness. Ablation studies confirm the effectiveness of each component, while visualization and cross-dataset evaluation further validate the model’s generalizability and resilience under complex, heterogeneous sensing conditions.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Hyperspectral image (HSI) and LiDAR data joint classification is a challenging task. Existing multi-source remote sensing data classification methods often rely on human-designed frameworks for feature extraction, which heavily depend on expert knowledge. To address these limitations, we propose a novel Dynamic Cross-Modal Feature Interaction Network (DCMNet), the first framework leveraging a dynamic routing mechanism for HSI and LiDAR classification. Specifically, our approach introduces three feature interaction blocks: Bilinear Spatial Attention Block (BSAB), Bilinear Channel Attention Block (BCAB), and Integration Convolutional Block (ICB). These blocks are designed to effectively enhance spatial, spectral, and discriminative feature interactions. A multi-layer routing space with routing gates is designed to determine optimal computational paths, enabling data-dependent feature fusion. Additionally, bilinear attention mechanisms are employed to enhance feature interactions in spatial and channel representations. Extensive experiments on three public HSI and LiDAR datasets demonstrate the superiority of DCMNet over state-of-the-art methods. Our code will be available at https://github.com/oucailab/DCMNet.

Problem

• Challenges in joint classification of hyperspectral and LiDAR data.
• Limitations of human-designed frameworks for feature extraction.
• Need for dynamic feature interaction in multi-source remote sensing.

Innovation

• Dynamic routing mechanism for feature fusion
• Bilinear attention enhances spatial-channel interactions
• Multi-layer routing space optimizes computational paths