Coded-E2LF: Coded Aperture Light Field Imaging from Events

2026-02-25

AI Summary

This work addresses the challenge of reconstructing four-dimensional (4D) light fields using only event camera data, proposing a purely event-driven computational imaging approach. By integrating an encoded aperture optical design with a static event camera, the method achieves, for the first time, pixel-level accurate reconstruction of 4D light fields directly from event streams. The study further uncovers the critical role of dark patterns within the encoded aperture, which substantially simplifies hardware requirements. Experimental validation on a real-world system demonstrates successful recovery of light fields from complex 3D scenes, confirming the feasibility and effectiveness of purely event-based light field imaging.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Machine Learning: Hardware-aware ML
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search

Abstract

We propose Coded-E2LF (coded event to light field), a computational imaging method for acquiring a 4-D light field using a coded aperture and a stationary event-only camera. In a previous work, an imaging system similar to ours was adopted, but both events and intensity images were captured and used for light field reconstruction. In contrast, our method is purely event-based, which relaxes restrictions for hardware implementation. We also introduce several advancements from the previous work that enable us to theoretically support and practically improve light field reconstruction from events alone. In particular, we clarify the key role of a black pattern in aperture coding patterns. We finally implemented our method on real imaging hardware to demonstrate its effectiveness in capturing real 3-D scenes. To the best of our knowledge, we are the first to demonstrate that a 4-D light field with pixel-level accuracy can be reconstructed from events alone. Our software and supplementary video are available from our project website.

Problem

• light field
• event camera
• coded aperture
• computational imaging
• 4D reconstruction

Innovation

• event-based vision
• light field reconstruction
• coded aperture
• computational imaging
• 4D light field