RACANet: Reliability-Aware Crowd Anchor Network for RGB-T Crowd Counting

2026-04-27

AI Summary

Existing RGB-T crowd counting methods lack explicit modeling of local spatial discrepancies and position-wise modality reliability during cross-modal fusion, limiting both accuracy and interpretability. To address this, this work proposes RACANet, a two-stage fusion framework. First, a lightweight cross-modal alignment pre-training guided by crowd priors enables local bidirectional soft matching. Subsequently, a Local Anchor Fusion Module (LAFM) adaptively redistributes features at the pixel level through semantic anchors and a local attention mechanism, augmented with a discrepancy-aware consistency constraint. Evaluated on the RGBT-CC and Drone-RGBT datasets, the proposed method significantly outperforms current state-of-the-art approaches, achieving enhanced counting accuracy and robustness.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

RGB-Thermal (T) crowd counting aims to integrate visible-spectrum and thermal infrared information to improve the robustness of crowd density estimation in complex scenes. Although existing studies generally improve counting accuracy through cross-modal feature fusion, most current methods rely on implicit cross-modal fusion strategies and lack explicit modeling of local spatial discrepancies as well as fine-grained characterization of modality reliability at the positional level, thereby limiting the accuracy and interpretability of the fusion process. To address these issues, this paper proposes a two-stage fusion framework, RACANet, a Reliability-Aware Crowd Anchor Network for RGB-T crowd counting. First, we introduce a lightweight cross-modal alignment pretraining stage, which explicitly learns cross-modal semantic correspondences through crowd-prior supervision and local bidirectional soft matching. Then, based on the priors learned during pretraining, a Local Anchor Fusion Module (LAFM) is introduced in the formal training stage. This module generates local semantic anchors by aggregating features from highly reliable regions and further enables adaptive pixel-level feature redistribution with a local attention mechanism. In addition, we propose a discrepancy-aware consistency constraint to dynamically coordinate the reliability of regions where modal representations are consistent. Experiments conducted on two widely used benchmark datasets, RGBT-CC and Drone-RGBT, demonstrate that RACANet outperforms existing methods. The anonymous code is available at https://anonymous.4open.science/r/RACANet-9985.

Problem

• RGB-T crowd counting
• cross-modal fusion
• modality reliability
• spatial discrepancy
• crowd density estimation

Innovation

• cross-modal alignment
• reliability-aware fusion
• local anchor
• RGB-T crowd counting
• discrepancy-aware consistency