Let There Be Light: Robust Lensless Imaging Under External Illumination With Deep Learning

2024-09-25

arXiv.org

AI Summary

Lensless imaging suffers severe degradation in complex ambient lighting conditions (e.g., fluorescent illumination, direct sunlight), leading to substantial image quality loss. To address this, we propose a robust reconstruction framework that synergistically integrates physical modeling with deep learning. Our method explicitly models external illumination noise and jointly estimates and suppresses it within a differentiable, learnable physical reconstruction architecture, enabling end-to-end optimization of image recovery and denoising. We further introduce an experimental calibration-based training strategy and publicly release the first large-scale real-world dataset—comprising 25K samples captured under diverse illumination conditions—for lensless imaging under ambient light. Extensive experiments demonstrate significant improvements: +3.2 dB in PSNR and +0.18 in SSIM over conventional approaches. All code and data are fully open-sourced.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Security and Privacy: Data transparency and provenance

Abstract

Lensless cameras relax the design constraints of traditional cameras by shifting image formation from analog optics to digital post-processing. While new camera designs and applications can be enabled, lensless imaging is very sensitive to unwanted interference (other sources, noise, etc.). In this work, we address a prevalent noise source that has not been studied for lensless imaging: external illumination e.g. from ambient and direct lighting. Being robust to a variety of lighting conditions would increase the practicality and adoption of lensless imaging. To this end, we propose multiple recovery approaches that account for external illumination by incorporating its estimate into the image recovery process. At the core is a physics-based reconstruction that combines learnable image recovery and denoisers, all of whose parameters are trained using experimentally gathered data. Compared to standard reconstruction methods, our approach yields significant qualitative and quantitative improvements. We open-source our implementations and a 25K dataset of measurements under multiple lighting conditions.

Problem

• Lensless camera
• Image quality degradation
• Variable lighting conditions

Innovation

• Lensless Camera
• Image Reconstruction
• Machine Learning