InfraDiffusion: zero-shot depth map restoration with diffusion models and prompted segmentation from sparse infrastructure point clouds

2025-09-03

AI Summary

In low-light scenarios such as masonry tunnels, acquiring high-resolution imagery is challenging, and sparse, noisy point clouds hinder brick-level fine-grained segmentation. Method: This paper proposes the first training-free, zero-shot depth map enhancement framework. It first projects the raw point cloud into an initial depth map via a virtual camera; then applies a Denoising Diffusion Null-space Model (DDNM) for zero-shot depth map restoration—improving geometric consistency and structural integrity; finally, integrates the Segment Anything Model (SAM) for prompt-driven, brick-level semantic segmentation. Contribution/Results: Experiments demonstrate substantial improvements in depth map quality and SAM segmentation accuracy. Evaluated on real-world bridge-and-tunnel point cloud data, the framework exhibits strong effectiveness and robustness against sparsity and noise. It establishes a novel paradigm for automated infrastructure defect detection without requiring labeled training data or domain-specific model adaptation.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

Point clouds are widely used for infrastructure monitoring by providing geometric information, where segmentation is required for downstream tasks such as defect detection. Existing research has automated semantic segmentation of structural components, while brick-level segmentation (identifying defects such as spalling and mortar loss) has been primarily conducted from RGB images. However, acquiring high-resolution images is impractical in low-light environments like masonry tunnels. Point clouds, though robust to dim lighting, are typically unstructured, sparse, and noisy, limiting fine-grained segmentation. We present InfraDiffusion, a zero-shot framework that projects masonry point clouds into depth maps using virtual cameras and restores them by adapting the Denoising Diffusion Null-space Model (DDNM). Without task-specific training, InfraDiffusion enhances visual clarity and geometric consistency of depth maps. Experiments on masonry bridge and tunnel point cloud datasets show significant improvements in brick-level segmentation using the Segment Anything Model (SAM), underscoring its potential for automated inspection of masonry assets. Our code and data is available at https://github.com/Jingyixiong/InfraDiffusion-official-implement.

Problem

• Restoring sparse infrastructure point clouds for segmentation
• Enabling brick-level defect detection in low-light environments
• Improving depth map clarity without task-specific training

Innovation

• Uses virtual cameras for depth map projection
• Adapts DDNM for zero-shot depth restoration
• Leverages SAM for enhanced brick-level segmentation