Prompt-Driven Building Footprint Extraction in Aerial Images With Offset-Building Model

2023-10-25

IEEE Transactions on Geoscience and Remote Sensing

AI Summary

To address the weak generalization and high human-in-the-loop cost in extracting unseen building footprints from ultra-high-resolution aerial imagery, this paper proposes a promptable end-to-end framework that jointly predicts roof segmentation and instance-level roof-to-footprint offset vectors. Key contributions include: (i) the first Offset-Building Model (OBM), explicitly modeling geometric displacement between roofs and footprints; (ii) distance-aware non-maximum suppression (DNMS) to enhance detection robustness under scale and density variations; and (iii) a prompt-based evaluation paradigm enabling efficient human–AI collaboration. Evaluated on the Huizhou test set (7,000+ samples), our method significantly outperforms baselines: offset error decreases by 16.6%, roof IoU improves by 10.8%, and offset vector loss drops by 6.5%. These results demonstrate superior generalization capability and practical engineering applicability for large-scale footprint extraction.

Abstract

More accurate extraction of invisible building footprints from very-high-resolution (VHR) aerial images relies on roof segmentation and roof-to-footprint offset extraction. Existing methods based on instance segmentation suffer from poor generalization when extended to large-scale data production and fail to achieve low-cost human interaction. This prompt paradigm inspires us to design a promptable framework for roof and offset extraction, and transforms end-to-end algorithms into promptable methods. Within this framework, we propose a novel offset-building model (OBM). Based on prompt prediction, we first discover a common pattern of predicting offsets and tailored Distance-NMS (DNMS) algorithms for offset optimization. To rigorously evaluate the algorithm’s capabilities, we introduce a prompt-based evaluation method, where our model reduces offset errors by 16.6% and improves roof Intersection over Union (IoU) by 10.8% compared to other models. Leveraging the common patterns in predicting offsets, DNMS algorithms enable models to further reduce offset vector loss (VL) by 6.5%. To further validate the generalization of models, we tested them using a newly proposed test set, Huizhou test set, with over 7,000 manually annotated instance samples. Our algorithms and dataset will be available at https://github.com/likaiucas/OBM.

Problem

• Accurate building footprint extraction
• Roof-to-footprint offset prediction
• Promptable framework for aerial images

Innovation

• Prompt-driven roof and offset extraction
• Offset-Building Model (OBM) introduced
• Distance-NMS algorithms optimize offset prediction