ME-CPT: Multi-Task Enhanced Cross-Temporal Point Transformer for Urban 3D Change Detection

2025-01-23

AI Summary

Addressing three key challenges in 3D semantic change detection for urban multi-temporal airborne LiDAR (ALS) point clouds—(1) difficulty in cross-temporal spatial modeling, (2) severe class imbalance among change categories, and (3) absence of large-scale, real-world benchmark datasets—this paper proposes: (1) a novel cross-temporal point cloud attention association mechanism enabling spatiotemporally consistent feature alignment; (2) a joint multi-task learning framework unifying semantic segmentation and change detection, integrated with a class-balanced loss; and (3) UrbanChange, the first large-scale, real-world urban 3D semantic change detection dataset covering 22.5 km². Extensive experiments demonstrate significant improvements over state-of-the-art methods across multiple benchmarks, particularly for underrepresented change classes such as construction and demolition. Both code and the UrbanChange dataset are fully open-sourced.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The point clouds collected by the Airborne Laser Scanning (ALS) system provide accurate 3D information of urban land covers. By utilizing multi-temporal ALS point clouds, semantic changes in urban area can be captured, demonstrating significant potential in urban planning, emergency management, and infrastructure maintenance. Existing 3D change detection methods struggle to efficiently extract multi-class semantic information and change features, still facing the following challenges: (1) the difficulty of accurately modeling cross-temporal point clouds spatial relationships for effective change feature extraction; (2) class imbalance of change samples which hinders distinguishability of semantic features; (3) the lack of real-world datasets for 3D semantic change detection. To resolve these challenges, we propose the Multi-task Enhanced Cross-temporal Point Transformer (ME-CPT) network. ME-CPT establishes spatiotemporal correspondences between point cloud across different epochs and employs attention mechanisms to jointly extract semantic change features, facilitating information exchange and change comparison. Additionally, we incorporate a semantic segmentation task and through the multi-task training strategy, further enhance the distinguishability of semantic features, reducing the impact of class imbalance in change types. Moreover, we release a 22.5 $km^2$ 3D semantic change detection dataset, offering diverse scenes for comprehensive evaluation. Experiments on multiple datasets show that the proposed MT-CPT achieves superior performance compared to existing state-of-the-art methods. The source code and dataset will be released upon acceptance at url{https://github.com/zhangluqi0209/ME-CPT}.

Problem

• 3D Change Detection
• Urban ALS Point Clouds
• Imbalanced Data

Innovation

• Multi-task Learning
• Cross-temporal Point Cloud Transformer
• 3D Change Detection