🤖 AI Summary
To address the challenging problem of salient object detection (SOD) in unaligned RGB-T image pairs—characterized by spatial misalignment, scale discrepancies, and viewpoint shifts—this paper proposes the first alignment-free, lightweight cross-modal collaborative perception framework. Our method innovatively integrates a thin-plate spline (TPS)-driven spatial self-correction mechanism with a semantic correlation constraint-based joint learning paradigm. We design a dual-stream MobileViT-Mamba architecture incorporating three key components: a TPS Alignment Module (TPSAM), a Semantic Correlation Constraint Module (SCCM), and a Cross-Modal Correlation Module (CMCM), enabling deep semantic alignment and efficient sequence modeling. Evaluated on multiple unaligned benchmarks, our approach achieves state-of-the-art performance among lightweight RGB-T SOD methods, significantly outperforming mainstream approaches reliant on manual registration. Moreover, it reduces model parameters and computational cost by over 40%.
📝 Abstract
Existing RGB-T salient object detection methods predominantly rely on manually aligned and annotated datasets, struggling to handle real-world scenarios with raw, unaligned RGB-T image pairs. In practical applications, due to significant cross-modal disparities such as spatial misalignment, scale variations, and viewpoint shifts, the performance of current methods drastically deteriorates on unaligned datasets. To address this issue, we propose an efficient RGB-T SOD method for real-world unaligned image pairs, termed Thin-Plate Spline-driven Semantic Correlation Learning Network (TPS-SCL). We employ a dual-stream MobileViT as the encoder, combined with efficient Mamba scanning mechanisms, to effectively model correlations between the two modalities while maintaining low parameter counts and computational overhead. To suppress interference from redundant background information during alignment, we design a Semantic Correlation Constraint Module (SCCM) to hierarchically constrain salient features. Furthermore, we introduce a Thin-Plate Spline Alignment Module (TPSAM) to mitigate spatial discrepancies between modalities. Additionally, a Cross-Modal Correlation Module (CMCM) is incorporated to fully explore and integrate inter-modal dependencies, enhancing detection performance. Extensive experiments on various datasets demonstrate that TPS-SCL attains state-of-the-art (SOTA) performance among existing lightweight SOD methods and outperforms mainstream RGB-T SOD approaches.