SUM Parts: Benchmarking Part-Level Semantic Segmentation of Urban Meshes

2025-03-19

AI Summary

Existing urban-scene semantic segmentation methods primarily focus on images or point clouds, while textured meshes—rich in both geometric and appearance information—remain underexplored, especially lacking part-level annotated benchmarks. To address this gap, we introduce SUM Parts, the first large-scale, part-level semantic segmentation benchmark for urban textured meshes, covering 2.5 km² and featuring 21 fine-grained part categories with dual annotations at both mesh-patch and texture-map levels. We establish the first systematic evaluation framework tailored to part-level segmentation on textured meshes, including an in-house interactive tool for joint patch-and-texture annotation, a unified pipeline for mesh-patch labeling, texture-mapping alignment, and 3D segmentation model evaluation. Both the dataset and benchmark are publicly released, significantly advancing mesh-native semantic understanding. We conduct comprehensive evaluations across diverse 3D segmentation and interactive annotation methods, providing rigorous performance analysis and establishing new baselines.

Technology Category

• Computer Vision: Segmentation
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Representation, semantic annotation, enhancement, enrichments, access and/or integration of a variety of data on the Web
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Semantic segmentation in urban scene analysis has mainly focused on images or point clouds, while textured meshes - offering richer spatial representation - remain underexplored. This paper introduces SUM Parts, the first large-scale dataset for urban textured meshes with part-level semantic labels, covering about 2.5 km2 with 21 classes. The dataset was created using our own annotation tool, which supports both face- and texture-based annotations with efficient interactive selection. We also provide a comprehensive evaluation of 3D semantic segmentation and interactive annotation methods on this dataset. Our project page is available at https://tudelft3d.github.io/SUMParts/.

Problem

• Lack of large-scale datasets for urban textured meshes.
• Underexplored part-level semantic segmentation in urban meshes.
• Need for efficient annotation tools for textured meshes.

Innovation

• First large-scale dataset for urban textured meshes
• Supports face- and texture-based annotations
• Comprehensive evaluation of 3D semantic segmentation