DragOSM: Extract Building Roofs and Footprints from Aerial Images by Aligning Historical Labels

2025-09-22

AI Summary

To address significant roof-footprint misalignment, ambiguous semantic boundaries, and positional inaccuracies and class limitations in legacy vector annotations (e.g., OpenStreetMap) for oblique remote sensing imagery, this paper proposes DragOSM—a diffusion-based interactive label alignment framework. Methodologically, it introduces “alignment tokens” to formulate annotation rectification as a Gaussian noise-driven dynamic denoising process, enabling the first joint correction and extraction of roof and footprint polygons. The approach integrates geographically aware alignment with stochastic perturbation training and introduces ReBO, a large-scale benchmark dataset of oblique aerial imagery. Experiments demonstrate that DragOSM substantially improves label alignment accuracy, facilitating high-fidelity urban building reconstruction. The code, pre-trained models, and the ReBO dataset are publicly released.

Technology Category

• Computer Vision: Segmentation
• Search and Optimization: Distributed Search
• Intelligent Robots: State Estimation

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Extracting polygonal roofs and footprints from remote sensing images is critical for large-scale urban analysis. Most existing methods rely on segmentation-based models that assume clear semantic boundaries of roofs, but these approaches struggle in off- nadir images, where the roof and footprint are significantly displaced, and facade pixels are fused with the roof boundary. With the increasing availability of open vector map annotations, e.g., OpenStreetMap, utilizing historical labels for off-nadir image annotation has become viable because remote sensing images are georeferenced once captured. However, these historical labels commonly suffer from significant positional discrepancies with new images and only have one annotation (roof or footprint), which fails to describe the correct structures of a building. To address these discrepancies, we first introduce a concept of an alignment token, which encodes the correction vector to guide the label correction. Based on this concept, we then propose Drag OpenStreetMap Labels (DragOSM), a novel model designed to align dislocated historical labels with roofs and footprints. Specifically, DragOSM formulates the label alignment as an interactive denoising process, modeling the positional discrepancy as a Gaussian distribution. During training, it learns to correct these errors by simulating misalignment with random Gaussian perturbations; during inference, it iteratively refines the positions of input labels. To validate our method, we further present a new dataset, Repairing Buildings in OSM (ReBO), comprising 179,265 buildings with both OpenStreetMap and manually corrected annotations across 5,473 images from 41 cities. Experimental results on ReBO demonstrate the effectiveness of DragOSM. Code, dataset, and trained models are publicly available at https://github.com/likaiucas/DragOSM.git.

Problem

• Aligning historical map labels with new aerial images despite positional discrepancies
• Correcting dislocated building roof and footprint annotations in off-nadir imagery
• Utilizing OpenStreetMap data for accurate building structure extraction from georeferenced images

Innovation

• Uses alignment tokens to encode correction vectors
• Formulates label alignment as interactive denoising process
• Learns to correct errors with Gaussian perturbations