RoofSeg: An edge-aware transformer-based network for end-to-end roof plane segmentation

2025-08-26

AI Summary

To address three key challenges in airborne LiDAR point cloud roof plane segmentation—lack of end-to-end modeling, weak discriminability of edge features, and ineffective integration of geometric priors into training—this paper proposes an end-to-end edge-aware Transformer network. The method employs learnable plane queries to drive hierarchical instance mask prediction, enabling truly end-to-end segmentation. An Edge-Aware Mask Module (EAMM) is introduced to fuse local geometric priors and enhance representation learning at roof boundaries. Furthermore, an adaptive weighted mask loss and a novel plane geometric loss are designed to explicitly enforce planarity completeness and geometric consistency. Experiments demonstrate that the proposed approach significantly outperforms existing traditional and deep learning methods in boundary accuracy, planar completeness, and overall segmentation quality.

Technology Category

• Computer Vision: Segmentation
• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Learning on the Edge & Model Compression

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Security and Privacy: Browser and end-point security

Abstract

Roof plane segmentation is one of the key procedures for reconstructing three-dimensional (3D) building models at levels of detail (LoD) 2 and 3 from airborne light detection and ranging (LiDAR) point clouds. The majority of current approaches for roof plane segmentation rely on the manually designed or learned features followed by some specifically designed geometric clustering strategies. Because the learned features are more powerful than the manually designed features, the deep learning-based approaches usually perform better than the traditional approaches. However, the current deep learning-based approaches have three unsolved problems. The first is that most of them are not truly end-to-end, the plane segmentation results may be not optimal. The second is that the point feature discriminability near the edges is relatively low, leading to inaccurate planar edges. The third is that the planar geometric characteristics are not sufficiently considered to constrain the network training. To solve these issues, a novel edge-aware transformer-based network, named RoofSeg, is developed for segmenting roof planes from LiDAR point clouds in a truly end-to-end manner. In the RoofSeg, we leverage a transformer encoder-decoder-based framework to hierarchically predict the plane instance masks with the use of a set of learnable plane queries. To further improve the segmentation accuracy of edge regions, we also design an Edge-Aware Mask Module (EAMM) that sufficiently incorporates planar geometric prior of edges to enhance its discriminability for plane instance mask refinement. In addition, we propose an adaptive weighting strategy in the mask loss to reduce the influence of misclassified points, and also propose a new plane geometric loss to constrain the network training.

Problem

• Segments roof planes from LiDAR point clouds end-to-end
• Improves edge region segmentation accuracy using geometric priors
• Incorporates plane geometric loss to constrain network training

Innovation

• Edge-aware transformer network for end-to-end segmentation
• Learnable plane queries with encoder-decoder framework
• Geometric loss and adaptive weighting for edge refinement