Recov-Vision: Linking Street View Imagery and Vision-Language Models for Post-Disaster Recovery

2025-09-24

AI Summary

This study addresses the need for fine-grained post-disaster assessment of building occupancy status (i.e., whether a structure is inhabited). To this end, we propose FacadeTrack—a novel framework that first performs viewpoint correction and aligns panoramic street-view videos with individual housing units to enable precise facade-to-unit mapping. Subsequently, it integrates a vision-language model with a rule-driven, two-stage decision mechanism, jointly enhancing discrimination accuracy and ensuring interpretability. The end-to-end pipeline is fully auditable, supporting error tracing and rigorous quality control. Evaluated on post-hurricane assessments from two major events, FacadeTrack achieves 0.927 precision, 0.781 recall, and 0.848 F1-score—significantly outperforming single-stage baselines. Its reliable, explainable outputs directly inform triage prioritization, infrastructure restoration planning, and equitable resource allocation in disaster response.

Technology Category

• Computer Vision: Motion & Tracking
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Intelligent Robots: State Estimation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Building-level occupancy after disasters is vital for triage, inspections, utility re-energization, and equitable resource allocation. Overhead imagery provides rapid coverage but often misses facade and access cues that determine habitability, while street-view imagery captures those details but is sparse and difficult to align with parcels. We present FacadeTrack, a street-level, language-guided framework that links panoramic video to parcels, rectifies views to facades, and elicits interpretable attributes (for example, entry blockage, temporary coverings, localized debris) that drive two decision strategies: a transparent one-stage rule and a two-stage design that separates perception from conservative reasoning. Evaluated across two post-Hurricane Helene surveys, the two-stage approach achieves a precision of 0.927, a recall of 0.781, and an F-1 score of 0.848, compared with the one-stage baseline at a precision of 0.943, a recall of 0.728, and an F-1 score of 0.822. Beyond accuracy, intermediate attributes and spatial diagnostics reveal where and why residual errors occur, enabling targeted quality control. The pipeline provides auditable, scalable occupancy assessments suitable for integration into geospatial and emergency-management workflows.

Problem

• Assessing building-level occupancy after disasters using street view imagery
• Linking panoramic video to parcels and rectifying views to facades
• Extracting interpretable attributes for transparent occupancy decision strategies

Innovation

• Linking panoramic video to parcels for alignment
• Rectifying street views to building facades
• Using language-guided framework for interpretable attributes