MCANet: A Multi-Scale Class-Specific Attention Network for Multi-Label Post-Hurricane Damage Assessment using UAV Imagery

2025-09-04

AI Summary

To address the challenges of multi-class damage identification, insufficient multi-scale feature extraction, and weak discrimination among visually similar or co-occurring classes in post-hurricane UAV imagery, this paper proposes a Multi-Scale Class-Specific Attention Network (MSCSAN). MSCSAN employs a hierarchical backbone built upon Res2Net and introduces a class-aware multi-head residual attention mechanism, enabling each branch to adaptively focus on salient regions at distinct spatial granularities—thereby enhancing fine-grained discriminability and model interpretability. Evaluated on the RescueNet dataset, MSCSAN achieves a mean Average Precision (mAP) of 91.75%; with an eight-head configuration, mAP improves to 92.35%. Notably, performance gains exceed 6% for challenging classes such as “Road Blocked”. Moreover, the framework supports pixel-level damage localization with interpretable visualizations.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Rapid and accurate post-hurricane damage assessment is vital for disaster response and recovery. Yet existing CNN-based methods struggle to capture multi-scale spatial features and to distinguish visually similar or co-occurring damage types. To address these issues, we propose MCANet, a multi-label classification framework that learns multi-scale representations and adaptively attends to spatially relevant regions for each damage category. MCANet employs a Res2Net-based hierarchical backbone to enrich spatial context across scales and a multi-head class-specific residual attention module to enhance discrimination. Each attention branch focuses on different spatial granularities, balancing local detail with global context. We evaluate MCANet on the RescueNet dataset of 4,494 UAV images collected after Hurricane Michael. MCANet achieves a mean average precision (mAP) of 91.75%, outperforming ResNet, Res2Net, VGG, MobileNet, EfficientNet, and ViT. With eight attention heads, performance further improves to 92.35%, boosting average precision for challenging classes such as Road Blocked by over 6%. Class activation mapping confirms MCANet's ability to localize damage-relevant regions, supporting interpretability. Outputs from MCANet can inform post-disaster risk mapping, emergency routing, and digital twin-based disaster response. Future work could integrate disaster-specific knowledge graphs and multimodal large language models to improve adaptability to unseen disasters and enrich semantic understanding for real-world decision-making.

Problem

• Assessing multi-scale hurricane damage from UAV imagery
• Distinguishing visually similar co-occurring damage types
• Improving spatial feature capture for damage classification

Innovation

• Multi-scale Res2Net backbone for spatial context
• Class-specific residual attention for damage discrimination
• Multi-head attention balancing local and global features