FGSGT: Saliency-Guided Siamese Network Tracker Based on Key Fine-Grained Feature Information for Thermal Infrared Target Tracking

2025-04-19

AI Summary

Thermal infrared (TIR) images suffer from low contrast and impoverished texture details, leading to weak discriminability, susceptibility to interference, and drift in visual trackers. To address these challenges, we propose SG-Siam—a fully end-to-end saliency-guided Siamese network. It employs fine-grained feature parallel learning convolutional blocks to extract discriminative local features; introduces a multi-level bilinear fusion module to enable cross-scale interaction among fine-grained features; and incorporates a residual refinement mechanism for saliency prediction jointly optimized with a saliency-constrained loss function, enabling attention-driven robust tracking. Extensive experiments demonstrate state-of-the-art performance: SG-Siam achieves top accuracy and success rates on PTB-TIR and LSOTB-TIR benchmarks. On VOT-TIR 2015 and 2017, it attains accuracy scores of 0.78 and 0.75, respectively—significantly outperforming existing methods.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Tracking, profiling, and countermeasures against them
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Thermal infrared (TIR) images typically lack detailed features and have low contrast, making it challenging for conventional feature extraction models to capture discriminative target characteristics. As a result, trackers are often affected by interference from visually similar objects and are susceptible to tracking drift. To address these challenges, we propose a novel saliency-guided Siamese network tracker based on key fine-grained feature infor-mation. First, we introduce a fine-grained feature parallel learning convolu-tional block with a dual-stream architecture and convolutional kernels of varying sizes. This design captures essential global features from shallow layers, enhances feature diversity, and minimizes the loss of fine-grained in-formation typically encountered in residual connections. In addition, we propose a multi-layer fine-grained feature fusion module that uses bilinear matrix multiplication to effectively integrate features across both deep and shallow layers. Next, we introduce a Siamese residual refinement block that corrects saliency map prediction errors using residual learning. Combined with deep supervision, this mechanism progressively refines predictions, ap-plying supervision at each recursive step to ensure consistent improvements in accuracy. Finally, we present a saliency loss function to constrain the sali-ency predictions, directing the network to focus on highly discriminative fi-ne-grained features. Extensive experiment results demonstrate that the pro-posed tracker achieves the highest precision and success rates on the PTB-TIR and LSOTB-TIR benchmarks. It also achieves a top accuracy of 0.78 on the VOT-TIR 2015 benchmark and 0.75 on the VOT-TIR 2017 benchmark.

Problem

• TIR images lack detail and contrast, hindering feature extraction
• Trackers face interference from similar objects and tracking drift
• Proposed tracker enhances fine-grained feature capture and saliency prediction

Innovation

• Fine-grained feature parallel learning block
• Multi-layer feature fusion via bilinear multiplication
• Siamese residual refinement with saliency loss