Density-aware Global-Local Attention Network for Point Cloud Segmentation

2024-11-30

arXiv.org

AI Summary

To address poor segmentation performance for tiny objects and few-shot classes in real-world point clouds, this paper proposes the Density-Aware Local-Global Attention Network (DALA-Net). Methodologically: (i) it introduces a novel density-adaptive sliding-window local attention mechanism that dynamically expands the receptive field of each point while preserving fine-grained details in dense regions; (ii) it models local neighborhoods as tokens and integrates them into a hierarchical global attention framework; (iii) it designs a class-response binary cross-entropy loss coupled with an intermediate-layer existence prediction module to mitigate learning bias arising from multi-scale variations and few-shot class imbalance. Evaluated on SemanticKITTI, S3DIS, and ScanObjectNN, DALA-Net achieves significant improvements in IoU and mAP for tiny objects and few-shot categories, setting new state-of-the-art results across multiple benchmarks.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Learning to Search
• Computer Vision: Segmentation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

3D point cloud segmentation has a wide range of applications in areas such as autonomous driving, augmented reality, virtual reality and digital twins. The point cloud data collected in real scenes often contain small objects and categories with small sample sizes, which are difficult to handle by existing networks. In this regard, we propose a point cloud segmentation network that fuses local attention based on density perception with global attention. The core idea is to increase the effective receptive field of each point while reducing the loss of information about small objects in dense areas. Specifically, we divide different sized windows for local areas with different densities to compute attention within the window. Furthermore, we consider each local area as an independent token for the global attention of the entire input. A category-response loss is also proposed to balance the processing of different categories and sizes of objects. In particular, we set up an additional fully connected layer in the middle of the network for prediction of the presence of object categories, and construct a binary cross-entropy loss to respond to the presence of categories in the scene. In experiments, our method achieves competitive results in semantic segmentation and part segmentation tasks on several publicly available datasets. Experiments on point cloud data obtained from complex real-world scenes filled with tiny objects also validate the strong segmentation capability of our method for small objects as well as small sample categories.

Problem

• Segments 3D point clouds with small objects and rare categories
• Reduces information loss in dense areas through density-aware attention
• Balances category processing with novel category-response loss function

Innovation

• Fuses local and global attention mechanisms
• Divides density-adaptive windows for local attention
• Uses category-response loss for class balancing