AIM2PC: Aerial Image to 3D Building Point Cloud Reconstruction

2025-03-24

AI Summary

Reconstructing complete 3D building point clouds from single-view aerial imagery remains challenging due to existing methods’ focus on rooftops only, the absence of full-building ground-truth datasets, and unreliable camera pose estimation. To address this, we propose the first end-to-end diffusion-based framework for generating full-building (roof + façade) point clouds directly from monocular aerial images. Our approach innovatively incorporates binary masks and Sobel edge maps as conditional guidance, employs a multi-modal feature concatenation architecture built upon the Centered Denoising Diffusion Probabilistic Model (CDPM), and introduces a camera-pose-driven progressive point cloud projection mechanism. To support rigorous evaluation, we construct and publicly release AIM2PC—the first aerial reconstruction dataset featuring precise camera poses and high-fidelity, full-building ground-truth point clouds. On AIM2PC, our method achieves, for the first time, geometrically complete and edge-preserving full-building point cloud reconstruction from a single image, significantly outperforming all baselines.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Security and Privacy: Data transparency and provenance

Abstract

Three-dimensional urban reconstruction of buildings from single-view images has attracted significant attention over the past two decades. However, recent methods primarily focus on rooftops from aerial images, often overlooking essential geometrical details. Additionally, there is a notable lack of datasets containing complete 3D point clouds for entire buildings, along with challenges in obtaining reliable camera pose information for aerial images. This paper addresses these challenges by presenting a novel methodology, AIM2PC , which utilizes our generated dataset that includes complete 3D point clouds and determined camera poses. Our approach takes features from a single aerial image as input and concatenates them with essential additional conditions, such as binary masks and Sobel edge maps, to enable more edge-aware reconstruction. By incorporating a point cloud diffusion model based on Centered denoising Diffusion Probabilistic Models (CDPM), we project these concatenated features onto the partially denoised point cloud using our camera poses at each diffusion step. The proposed method is able to reconstruct the complete 3D building point cloud, including wall information and demonstrates superior performance compared to existing baseline techniques. To allow further comparisons with our methodology the dataset has been made available at https://github.com/Soulaimene/AIM2PCDataset

Problem

• Reconstructs 3D building point clouds from single aerial images
• Addresses lack of complete 3D datasets and camera pose challenges
• Improves edge-aware reconstruction with diffusion models and multi-feature inputs

Innovation

• Uses complete 3D point clouds dataset
• Incorporates edge-aware reconstruction conditions
• Applies CDPM-based point cloud diffusion model