A hierarchical semantic segmentation framework for computer vision-based bridge damage detection

2022-07-18

arXiv.org

AI Summary

Detecting fine-scale structural damages (e.g., micro-cracks, exposed rebar) in remote bridge imagery remains challenging due to low visual saliency and severe class imbalance—particularly among structural components exhibiting long-tailed distributions. Method: We propose a hierarchical semantic-constrained fine-grained damage segmentation framework. It jointly models structural component categories and damage types via a novel hierarchical semantic constraint mechanism. To address data sparsity and scale mismatch, we integrate category-aware importance sampling with multi-scale image augmentation. Building upon a deep semantic segmentation network, our approach incorporates hierarchical mask inference and multi-scale feature fusion. Results: Evaluated on a real-world bridge dataset, the method achieves a 12.6% improvement in crack mIoU and a 27.3% gain in recall for long-tailed classes. It significantly enhances robustness and generalization for small-object damage detection.

Abstract

Computer vision-based damage detection using remote cameras and unmanned aerial vehicles (UAVs) enables efficient and low-cost bridge health monitoring that reduces labor costs and the needs for sensor installation and maintenance. By leveraging recent semantic image segmentation approaches, we are able to find regions of critical structural components and recognize damage at the pixel level using images as the only input. However, existing methods perform poorly when detecting small damages (e.g., cracks and exposed rebars) and thin objects with limited image samples, especially when the components of interest are highly imbalanced. To this end, this paper introduces a semantic segmentation framework that imposes the hierarchical semantic relationship between component category and damage types. For example, certain concrete cracks only present on bridge columns and therefore the non-column region will be masked out when detecting such damages. In this way, the damage detection model could focus on learning features from possible damaged regions only and avoid the effects of other irrelevant regions. We also utilize multi-scale augmentation that provides views with different scales that preserves contextual information of each image without losing the ability of handling small and thin objects. Furthermore, the proposed framework employs important sampling that repeatedly samples images containing rare components (e.g., railway sleeper and exposed rebars) to provide more data samples, which addresses the imbalanced data challenge.

Problem

• Image Processing
• Bridge Damage Detection
• Small Defects Recognition

Innovation

• Hierarchical Semantic Segmentation
• Multi-scale Image Processing
• Imbalanced Data Handling