TrueCity: Real and Simulated Urban Data for Cross-Domain 3D Scene Understanding

2025-11-10

AI Summary

Real-world annotated 3D point clouds for semantic scene understanding are scarce, while synthetic data suffer from domain shift and lack synchronized real-sim paired benchmarks. Method: This paper introduces TrueCity—the first city-scale semantic segmentation benchmark—comprising centimeter-accurate real LiDAR point clouds, semantically annotated 3D urban models, and rigorously co-registered synthetic point clouds, all annotated under an ISO-compliant fine-grained semantic taxonomy. Contribution/Results: TrueCity enables the first quantitative analysis of Sim2Real domain adaptation and cross-domain semantic segmentation. Leveraging this benchmark, we systematically evaluate domain shift characteristics of state-of-the-art methods and empirically validate that synthetic data augmentation significantly improves generalization to real-world scenes. TrueCity thus provides a critical data foundation and evaluation platform for developing generalizable 3D perception models.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: 3D Computer Vision
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

3D semantic scene understanding remains a long-standing challenge in the 3D computer vision community. One of the key issues pertains to limited real-world annotated data to facilitate generalizable models. The common practice to tackle this issue is to simulate new data. Although synthetic datasets offer scalability and perfect labels, their designer-crafted scenes fail to capture real-world complexity and sensor noise, resulting in a synthetic-to-real domain gap. Moreover, no benchmark provides synchronized real and simulated point clouds for segmentation-oriented domain shift analysis. We introduce TrueCity, the first urban semantic segmentation benchmark with cm-accurate annotated real-world point clouds, semantic 3D city models, and annotated simulated point clouds representing the same city. TrueCity proposes segmentation classes aligned with international 3D city modeling standards, enabling consistent evaluation of synthetic-to-real gap. Our extensive experiments on common baselines quantify domain shift and highlight strategies for exploiting synthetic data to enhance real-world 3D scene understanding. We are convinced that the TrueCity dataset will foster further development of sim-to-real gap quantification and enable generalizable data-driven models. The data, code, and 3D models are available online: https://tum-gis.github.io/TrueCity/

Problem

• Addressing limited real-world annotated data for 3D semantic scene understanding
• Bridging the synthetic-to-real domain gap in urban point cloud segmentation
• Providing synchronized real and simulated urban data for domain shift analysis

Innovation

• TrueCity benchmark provides synchronized real and simulated urban point clouds
• It introduces cm-accurate annotated real-world data with semantic 3D models
• Dataset enables consistent evaluation of synthetic-to-real domain gap