Seeing the Unseen in Low-light Spike Streams

2025-09-27

AI Summary

To address severe noise and extreme signal sparsity in event-based high-speed imaging under low-light conditions, this paper proposes Diff-SPK—the first work to introduce diffusion models into spike-based camera reconstruction. Methodologically, we design an Event-Time Feature Integration (ETFI) module that explicitly encodes inter-spike temporal intervals and aggregates sparse cross-frame signals; we further incorporate a ControlNet architecture to enable generative prior-guided conditional reconstruction. Our contributions are threefold: (1) a novel diffusion-based generative prior modeling paradigm for spike stream reconstruction; (2) the first real-world low-light spike stream benchmark dataset; and (3) state-of-the-art performance—demonstrating significant quantitative and qualitative improvements over existing methods, with reconstructed images exhibiting enhanced clarity, richer texture preservation, and more accurate motion detail recovery.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Content-based information diffusion
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Spike camera, a type of neuromorphic sensor with high-temporal resolution, shows great promise for high-speed visual tasks. Unlike traditional cameras, spike camera continuously accumulates photons and fires asynchronous spike streams. Due to unique data modality, spike streams require reconstruction methods to become perceptible to the human eye. However, lots of methods struggle to handle spike streams in low-light high-speed scenarios due to severe noise and sparse information. In this work, we propose Diff-SPK, the first diffusion-based reconstruction method for spike camera. Diff-SPK effectively leverages generative priors to supplement texture information in low-light conditions. Specifically, it first employs an extbf{E}nhanced extbf{T}exture extbf{f}rom Inter-spike extbf{I}nterval (ETFI) to aggregate sparse information from low-light spike streams. Then, ETFI serves as a conditioning input for ControlNet to generate the high-speed scenes. To improve the quality of results, we introduce an ETFI-based feature fusion module during the generation process. Moreover, we establish the first bona fide benchmark for the low-light spike stream reconstruction task. It significantly surpasses existing reconstruction datasets in scale and provides quantitative illumination information. The performance on real low-light spike streams demonstrates the superiority of Diff-SPK.

Problem

• Reconstructing low-light spike streams with severe noise
• Handling sparse information in high-speed visual scenarios
• Generating perceptible images from asynchronous spike data

Innovation

• Uses diffusion model for spike stream reconstruction
• Employs ETFI to aggregate sparse spike information
• Integrates ControlNet with ETFI feature fusion module