RENO: Real-Time Neural Compression for 3D LiDAR Point Clouds

2025-03-16

AI Summary

To address the challenge of real-time compression of 3D LiDAR point clouds, this paper proposes RENO—the first lightweight, end-to-end neural codec. To avoid the high computational overhead of conventional octree-based representations, RENO introduces a novel sparse occupancy coding mechanism: it jointly predicts voxel occupancy states in a single inference pass via multi-scale sparse tensor representation and learnable encoding, achieving an efficient rate-distortion trade-off. The model employs a highly compact network architecture with only 1 MB of parameters. Evaluated on an RTX 3090 GPU, RENO achieves real-time encoding/decoding at 10 fps. Compared to G-PCCv23 and Draco, it reduces bitrate by 12.25% and 48.34%, respectively, while maintaining competitive reconstruction quality. These advances significantly enhance practical deployability in resource-constrained industrial applications such as autonomous driving and robotics.

Technology Category

• Computer Vision: 3D Computer Vision
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Learning on the Edge & Model Compression

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Sustainability and carbon-aware systems for Web, mobile, and WoT
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work

Abstract

Despite the substantial advancements demonstrated by learning-based neural models in the LiDAR Point Cloud Compression (LPCC) task, realizing real-time compression - an indispensable criterion for numerous industrial applications - remains a formidable challenge. This paper proposes RENO, the first real-time neural codec for 3D LiDAR point clouds, achieving superior performance with a lightweight model. RENO skips the octree construction and directly builds upon the multiscale sparse tensor representation. Instead of the multi-stage inferring, RENO devises sparse occupancy codes, which exploit cross-scale correlation and derive voxels' occupancy in a one-shot manner, greatly saving processing time. Experimental results demonstrate that the proposed RENO achieves real-time coding speed, 10 fps at 14-bit depth on a desktop platform (e.g., one RTX 3090 GPU) for both encoding and decoding processes, while providing 12.25% and 48.34% bit-rate savings compared to G-PCCv23 and Draco, respectively, at a similar quality. RENO model size is merely 1MB, making it attractive for practical applications. The source code is available at https://github.com/NJUVISION/RENO.

Problem

• Real-time compression for 3D LiDAR point clouds
• Lightweight neural codec with superior performance
• Efficient sparse occupancy codes for faster processing

Innovation

• Real-time neural codec for 3D LiDAR
• Uses multiscale sparse tensor representation
• One-shot sparse occupancy codes save time