LMSeg: An end-to-end geometric message-passing network on barycentric dual graphs for large-scale landscape mesh segmentation

2024-07-05

AI Summary

To address scalability limitations, difficulties in end-to-end training, and low recognition accuracy for small-scale or irregular terrain features (e.g., dry-stone walls) in large-scale 3D landscape mesh semantic segmentation, this paper proposes a face-centered dual-graph representation and a centroid-coordinate-based dual-graph message-passing network. We design a learnable geometric aggregation plus (GA+) module alongside a hierarchical-local dual pooling mechanism to jointly model high-frequency geometric details and global contextual information. The architecture is lightweight, with only 2.4M parameters. Evaluated on three major benchmarks—SUM, H3D, and BBW—the method achieves 75.1% mIoU, 78.4% overall accuracy (O.A.), and 62.4% mIoU, respectively. Notably, it significantly improves segmentation accuracy for cultural heritage features under heavy vegetation occlusion.

Technology Category

• Computer Vision: Segmentation
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Semantic segmentation of large-scale 3D landscape meshes is critical for geospatial analysis in complex environments, yet existing approaches face persistent challenges of scalability, end-to-end trainability, and accurate segmentation of small and irregular objects. To address these issues, we introduce the BudjBim Wall (BBW) dataset, a large-scale annotated mesh dataset derived from high-resolution LiDAR scans of the UNESCO World Heritage-listed Budj Bim cultural landscape in Victoria, Australia. The BBW dataset captures historic dry-stone wall structures that are difficult to detect under vegetation occlusion, supporting research in underrepresented cultural heritage contexts. Building on this dataset, we propose LMSeg, a deep graph message-passing network for semantic segmentation of large-scale meshes. LMSeg employs a barycentric dual graph representation of mesh faces and introduces the Geometry Aggregation+ (GA+) module, a learnable softmax-based operator that adaptively combines neighborhood features and captures high-frequency geometric variations. A hierarchical-local dual pooling integrates hierarchical and local geometric aggregation to balance global context with fine-detail preservation. Experiments on three large-scale benchmarks (SUM, H3D, and BBW) show that LMSeg achieves 75.1% mIoU on SUM, 78.4% O.A. on H3D, and 62.4% mIoU on BBW, using only 2.4M lightweight parameters. In particular, LMSeg demonstrates accurate segmentation across both urban and natural scenes-capturing small-object classes such as vehicles and high vegetation in complex city environments, while also reliably detecting dry-stone walls in dense, occluded rural landscapes. Together, the BBW dataset and LMSeg provide a practical and extensible method for advancing 3D mesh segmentation in cultural heritage, environmental monitoring, and urban applications.

Problem

• Addresses semantic segmentation of large-scale 3D landscape meshes
• Solves scalability and end-to-end trainability in complex environments
• Improves segmentation accuracy for small irregular objects under occlusion

Innovation

• End-to-end geometric message-passing network on barycentric dual graphs
• Geometry Aggregation+ module adaptively combines neighborhood features
• Hierarchical-local dual pooling balances global context with detail preservation