Spectral Discrepancy and Cross-modal Semantic Consistency Learning for Object Detection in Hyperspectral Image

2025-12-20

AI Summary

Hyperspectral image target detection faces dual challenges: spatial misalignment across spectral bands and spectral ambiguity arising from intra-class variability and inter-class similarity. To address these, we propose an end-to-end framework featuring three key innovations: (1) a Spectral Difference Awareness (SDA) module that explicitly models inter-band spectral variation; (2) a Spectral Gating Generator (SGG) that adaptively suppresses redundant bands while enhancing pixel-level spectral discriminability; and (3) a Semantic Consistency Learning (SCL) mechanism that jointly optimizes multi-band feature alignment and cross-modal semantic consistency. Evaluated on two standard hyperspectral benchmarks, our method achieves state-of-the-art performance in fine-grained material identification and small-target detection, with significant improvements in both accuracy and robustness.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Semi-Supervised Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Hyperspectral images with high spectral resolution provide new insights into recognizing subtle differences in similar substances. However, object detection in hyperspectral images faces significant challenges in intra- and inter-class similarity due to the spatial differences in hyperspectral inter-bands and unavoidable interferences, e.g., sensor noises and illumination. To alleviate the hyperspectral inter-bands inconsistencies and redundancy, we propose a novel network termed extbf{S}pectral extbf{D}iscrepancy and extbf{C}ross- extbf{M}odal semantic consistency learning (SDCM), which facilitates the extraction of consistent information across a wide range of hyperspectral bands while utilizing the spectral dimension to pinpoint regions of interest. Specifically, we leverage a semantic consistency learning (SCL) module that utilizes inter-band contextual cues to diminish the heterogeneity of information among bands, yielding highly coherent spectral dimension representations. On the other hand, we incorporate a spectral gated generator (SGG) into the framework that filters out the redundant data inherent in hyperspectral information based on the importance of the bands. Then, we design the spectral discrepancy aware (SDA) module to enrich the semantic representation of high-level information by extracting pixel-level spectral features. Extensive experiments on two hyperspectral datasets demonstrate that our proposed method achieves state-of-the-art performance when compared with other ones.

Problem

• Addresses intra- and inter-class similarity challenges in hyperspectral object detection
• Alleviates spectral band inconsistencies and redundancy for improved feature extraction
• Enhances semantic representation by learning cross-modal consistency and spectral features

Innovation

• Spectral discrepancy aware module extracts pixel-level spectral features
• Spectral gated generator filters redundant hyperspectral data
• Semantic consistency learning reduces inter-band information heterogeneity