SP-TransientBench: A Real-Captured Single Photon Perception Benchmark

2026-06-16

AI Summary

This work addresses the significant challenges in geometric reconstruction and semantic understanding under real-world single-photon sensing, which are exacerbated by unique noise characteristics and multi-path transient effects, compounded by a longstanding absence of realistic benchmark data. To bridge this gap, we present the first large-scale, real-captured single-photon LiDAR multitask benchmark, acquired using a solid-state single-photon sensor (256×192) that records full time-of-flight histograms. The dataset includes precise poses, multi-view calibration, and 3D semantic annotations across 13 classes, spanning 10 scenes and 10,297 viewpoints. It provides standardized data splits and evaluation protocols to enable reproducible, systematic research on tasks such as depth estimation, multi-view reconstruction, and 3D semantic understanding, thereby substantially advancing real-world single-photon 3D vision.

Abstract

Single-photon LiDAR (SPL) based on single-photon avalanche diode (SPAD) sensing enables time-resolved photon measurements with extreme sensitivity, offering unique potential for active 3D perception in photon-starved scenarios.However, real-world single photon perception remains fundamentally challenging due to unique measurement noise and complex multi-return transient phenomena, which jointly complicate geometric reconstruction and semantic scene understanding. Despite growing interest in SPAD-based sensing, existing studies are largely limited to simulated data or small-scale controlled captures. As a result, systematic evaluation of real-world single photon perception across depth estimation, multi-view reconstruction, and 3D semantic understanding remains underexplored. To bridge this gap, we introduce SP-TransientBench (STB), a real-captured multi-task benchmark for single photon perception. SP-TransientBenc comprises 10 diverse scenes and 10,297 views captured using a solid-state single-photon LiDAR at $256\times192$ resolution. Each view provides full time-of-flight histograms with multi-return behavior,standardized metadata, and calibrated camera poses for multi-view evaluation. We further provide 13-class 3D semantic annotations for selected scenes. By providing dedicated data splits and evaluation protocols for each task, STB enables consistent and reproducible benchmarking of real-world single photon perception across multiple 3D vision problems. The dataset and code will be released upon acceptance.

Problem

• single-photon perception
• real-world benchmark
• multi-return transient
• 3D semantic understanding
• SPAD-based sensing

Innovation

• single-photon LiDAR
• SPAD sensing
• multi-return transient
• real-captured benchmark
• 3D semantic understanding