Accurate Simulation Pipeline for Passive Single-Photon Imaging

2026-01-19

IEEE Sensors Journal

AI Summary

This work addresses the high cost and scarcity of SPAD sensor data, which severely hinder the development of single-photon imaging algorithms and deep learning models. To overcome this limitation, the authors present a high-fidelity, physics-based simulation pipeline for passive single-photon imaging that generates multimodal synthetic data. They introduce SPAD-MNIST, the first publicly available dataset of its kind. Cross-domain validation on real SPAD hardware demonstrates that a CNN classifier trained exclusively on simulated data maintains excellent performance even under extremely low illumination (5 mlux), confirming the accuracy and practical utility of the proposed simulation framework. This approach provides a robust foundation of data and methodology for advancing vision tasks in ultra-low-light conditions.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Machine Learning: Multimodal Learning
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Web Mining and Content Analysis: Web data generation and simulation
• Security and Privacy: Data transparency and provenance
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Single-photon avalanche diodes (SPADs) are new and promising imaging sensors. These sensors are sensitive enough to detect individual photons hitting each pixel, with extreme temporal resolution and without readout noise. Thus, SPADs stand out as an optimal choice for lowlight imaging. Due to the high price and limited availability of SPAD sensors, the demand for an accurate data simulation pipeline is substantial. Indeed, the scarcity of SPAD datasets hinders the development of SPAD-specific processing algorithms and impedes the training of learning-based solutions. In this article, we present a comprehensive SPAD simulation pipeline and validate it with multiple experiments using two recent commercial SPAD sensors. Our simulator is used to generate the SPAD-MNIST, a single-photon version of the seminal MNIST dataset, to investigate the effectiveness of convolutional neural network (CNN) classifiers on reconstructed fluxes, even at extremely low light conditions, e.g., 5 mlx. We also assess the performance of classifiers exclusively trained on simulated data on real images acquired from SPAD sensors at different light conditions. The synthetic dataset encompasses different SPAD imaging modalities and is made available for download. Project page: https://boracchi.faculty.polimi.it/Projects/SPAD-MNIST.html

Problem

• SPAD
• single-photon imaging
• data simulation
• low-light imaging
• dataset scarcity

Innovation

• SPAD simulation
• single-photon imaging
• low-light vision
• synthetic dataset
• CNN classification