eBIRD: Event-based Intensity Image Reconstruction Using Controllable Diffusion Models

2026-08-09

AI Summary

Reconstructing intensity images from event streams is highly challenging due to the binary, sparse, and asynchronous nature of event data. This work proposes eBIRD, a novel framework that, for the first time, applies a ControlNet-guided denoising diffusion probabilistic model (DDPM) to event-based image reconstruction, leveraging a 33-ms event window for conditional generation. The study systematically evaluates the adaptability of general-purpose versus specialized diffusion learning strategies across domains and achieves state-of-the-art reconstruction performance on both N-MNIST (MSE: 0.0052, SSIM: 0.8982, PSNR: 23.34 dB) and RGBE-Gaze (MSE: 0.0161, SSIM: 0.7605, PSNR: 19.08 dB) benchmarks.

Abstract

Intensity-image reconstruction from event streams remains a challenging problem due to the binary, sparse, and asynchronous nature of event data. This work proposes eBIRD, an event-guided reconstruction framework that combines a DDPM with ControlNet-based conditioning. We analyze generic and specialized diffusion learning strategies for handwritten digit (N-MNIST) and face (RGBE-Gaze) reconstruction using 33ms event windows. On N-MNIST, the general model achieves the best reconstruction quality (MSE 0.0052, SSIM 0.8982, PSNR 23.34dB), whereas the specialized model performs best on RGBE-Gaze (MSE 0.0161, SSIM 0.7605, PSNR 19.08dB). These preliminary results suggest that controllable diffusion models are a promising approach for event-guided intensity-image reconstruction, while highlighting that the preferred learning strategy depends on the reconstruction domain.

Problem

• event-based vision
• intensity image reconstruction
• event streams
• asynchronous data
• sparse data

Innovation

• event-based vision
• diffusion models
• ControlNet
• image reconstruction
• spiking data