Robust 3D Object Detection using Probabilistic Point Clouds from Single-Photon LiDARs

2025-07-31

AI Summary

Single-photon LiDAR often yields sparse and noisy point clouds under challenging conditions—such as long ranges, low surface reflectivity, and strong ambient illumination—while conventional 3D processing pipelines neglect the inherent uncertainty in raw measurements, degrading downstream detection robustness. To address this, we propose a probabilistic point cloud representation that explicitly models per-point measurement confidence by jointly encoding raw photon counts and geometric observations into point clouds augmented with probabilistic attributes. We further design a lightweight uncertainty-aware inference module, compatible as a plug-and-play component within mainstream 3D detection frameworks. Extensive experiments across diverse indoor and outdoor scenarios demonstrate significant improvements in detecting small and distant objects, outperforming both standard LiDAR-based and camera-LiDAR fusion approaches. Our results empirically validate that explicit uncertainty modeling is critical for enhancing robustness in 3D perception.

Technology Category

• Reasoning under Uncertainty: Uncertainty Representations
• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: 3D Computer Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

LiDAR-based 3D sensors provide point clouds, a canonical 3D representation used in various scene understanding tasks. Modern LiDARs face key challenges in several real-world scenarios, such as long-distance or low-albedo objects, producing sparse or erroneous point clouds. These errors, which are rooted in the noisy raw LiDAR measurements, get propagated to downstream perception models, resulting in potentially severe loss of accuracy. This is because conventional 3D processing pipelines do not retain any uncertainty information from the raw measurements when constructing point clouds. We propose Probabilistic Point Clouds (PPC), a novel 3D scene representation where each point is augmented with a probability attribute that encapsulates the measurement uncertainty (or confidence) in the raw data. We further introduce inference approaches that leverage PPC for robust 3D object detection; these methods are versatile and can be used as computationally lightweight drop-in modules in 3D inference pipelines. We demonstrate, via both simulations and real captures, that PPC-based 3D inference methods outperform several baselines using LiDAR as well as camera-LiDAR fusion models, across challenging indoor and outdoor scenarios involving small, distant, and low-albedo objects, as well as strong ambient light. Our project webpage is at https://bhavyagoyal.github.io/ppc .

Problem

• Address sparse or erroneous LiDAR point clouds
• Retain uncertainty in raw LiDAR measurements
• Improve 3D object detection in challenging scenarios

Innovation

• Probabilistic Point Clouds with uncertainty attributes
• Lightweight drop-in modules for 3D inference
• Outperforms LiDAR and fusion baselines