FRNet: Frustum-Range Networks for Scalable LiDAR Segmentation

2023-12-07

arXiv.org

AI Summary

To address context information loss and excessive reliance on post-processing in range-view LiDAR semantic segmentation, this paper proposes FRNet. First, it introduces a novel frustum feature encoder that jointly models range images and their corresponding 3D frustum-aligned point cloud features. Second, it designs a hierarchical frustum-point fusion module to enhance local contextual representation while preserving point-wise prediction consistency. Third, it incorporates a multi-scale head fusion mechanism to improve semantic discrimination. Evaluated on SemanticKITTI and nuScenes, FRNet achieves 73.3% and 82.5% mIoU, respectively—surpassing prior state-of-the-art methods. Moreover, it operates five times faster than the current SOTA, striking a superior balance among accuracy, real-time inference capability, and architectural scalability.

Abstract

LiDAR segmentation has become a crucial component of advanced autonomous driving systems. Recent range-view LiDAR segmentation approaches show promise for real-time processing. However, they inevitably suffer from corrupted contextual information and rely heavily on post-processing techniques for prediction refinement. In this work, we propose FRNet, a simple yet powerful method aimed at restoring the contextual information of range image pixels using corresponding frustum LiDAR points. First, a frustum feature encoder module is used to extract per-point features within the frustum region, which preserves scene consistency and is critical for point-level predictions. Next, a frustum-point fusion module is introduced to update per-point features hierarchically, enabling each point to extract more surrounding information through the frustum features. Finally, a head fusion module is used to fuse features at different levels for final semantic predictions. Extensive experiments conducted on four popular LiDAR segmentation benchmarks under various task setups demonstrate the superiority of FRNet. Notably, FRNet achieves 73.3% and 82.5% mIoU scores on the testing sets of SemanticKITTI and nuScenes. While achieving competitive performance, FRNet operates 5 times faster than state-of-the-art approaches. Such high efficiency opens up new possibilities for more scalable LiDAR segmentation. The code has been made publicly available at https://github.com/Xiangxu-0103/FRNet.

Problem

• Restores contextual information in LiDAR segmentation
• Improves efficiency and speed of LiDAR processing
• Enhances semantic predictions using frustum-point fusion

Innovation

• Frustum feature encoder preserves scene consistency
• Frustum-point fusion updates per-point features hierarchically
• Head fusion module combines features for semantic predictions