A Benchmark Dataset for Spatially Aligned Road Damage Assessment in Small Uncrewed Aerial Systems Disaster Imagery

2025-12-12

AI Summary

Existing post-disaster road damage assessment suffers from a lack of large-scale, high-resolution benchmark datasets, while severe spatial misalignment between remote sensing imagery and reference road networks degrades model performance—reducing average Macro IoU by 5.6% and causing erroneous labeling of ~11 km of disaster-affected road segments. Method: We introduce the largest publicly available benchmark for post-disaster road damage assessment with explicit spatial alignment: it encompasses sUAS imagery from 10 federally declared disasters, annotates 657.25 km of roads, and corrects 9,184 instances of road network displacement. We systematically quantify the impact of geospatial misregistration and propose a novel spatial alignment paradigm tailored for emergency response. Our framework employs 10 fine-grained damage classes, integrates 18 ML/CV models, and combines geographic registration with manual refinement for high-precision alignment. Results: The models have been operationally deployed during Hurricanes Debby and Helene, reducing通行ability assessment error by 8–9%.

Technology Category

• Computer Vision: Remote Sensing / Geospatial AI
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Intelligent Robots: Localization, Mapping, and Navigation

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

This paper presents the largest known benchmark dataset for road damage assessment and road alignment, and provides 18 baseline models trained on the CRASAR-U-DRIODs dataset's post-disaster small uncrewed aerial systems (sUAS) imagery from 10 federally declared disasters, addressing three challenges within prior post-disaster road damage assessment datasets. While prior disaster road damage assessment datasets exist, there is no current state of practice, as prior public datasets have either been small-scale or reliant on low-resolution imagery insufficient for detecting phenomena of interest to emergency managers. Further, while machine learning (ML) systems have been developed for this task previously, none are known to have been operationally validated. These limitations are overcome in this work through the labeling of 657.25km of roads according to a 10-class labeling schema, followed by training and deploying ML models during the operational response to Hurricanes Debby and Helene in 2024. Motivated by observed road line misalignment in practice, 9,184 road line adjustments were provided for spatial alignment of a priori road lines, as it was found that when the 18 baseline models are deployed against real-world misaligned road lines, model performance degraded on average by 5.596% Macro IoU. If spatial alignment is not considered, approximately 8% (11km) of adverse conditions on road lines will be labeled incorrectly, with approximately 9% (59km) of road lines misaligned off the actual road. These dynamics are gaps that should be addressed by the ML, CV, and robotics communities to enable more effective and informed decision-making during disasters.

Problem

• Creating a benchmark dataset for road damage assessment from sUAS imagery
• Addressing spatial misalignment of road lines in disaster imagery analysis
• Providing operationally validated ML models for emergency management decisions

Innovation

• Largest benchmark dataset for road damage assessment
• 18 baseline models trained on post-disaster sUAS imagery
• Spatial alignment corrections to improve model accuracy