Towards Robust and Generalizable Lensless Imaging with Modular Learned Reconstruction

2025-02-03

AI Summary

Lensless cameras suffer from poor generalization across mask types and hardware platforms, coupled with prohibitively long training cycles (weeks) for new systems. Method: We propose a modular learning-based reconstruction framework featuring a novel decoupled preprocessor–reconstructor architecture, theoretically establishing the necessity of a physics-informed front-end preprocessor for standard reconstruction algorithms. Our design integrates Wiener filtering, iterative optimization, and deep neural networks to enable component reuse and transfer learning. Contribution/Results: We introduce the first cross-amplitude/phase-mask generalization benchmark. Experiments demonstrate significantly improved robustness to unseen mask types and reduce system calibration and training time from weeks to hours. We publicly release four high-quality datasets and a complete open-source software–hardware stack covering the entire pipeline.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Data transparency and provenance

Abstract

Lensless cameras disregard the conventional design that imaging should mimic the human eye. This is done by replacing the lens with a thin mask, and moving image formation to the digital post-processing. State-of-the-art lensless imaging techniques use learned approaches that combine physical modeling and neural networks. However, these approaches make simplifying modeling assumptions for ease of calibration and computation. Moreover, the generalizability of learned approaches to lensless measurements of new masks has not been studied. To this end, we utilize a modular learned reconstruction in which a key component is a pre-processor prior to image recovery. We theoretically demonstrate the pre-processor's necessity for standard image recovery techniques (Wiener filtering and iterative algorithms), and through extensive experiments show its effectiveness for multiple lensless imaging approaches and across datasets of different mask types (amplitude and phase). We also perform the first generalization benchmark across mask types to evaluate how well reconstructions trained with one system generalize to others. Our modular reconstruction enables us to use pre-trained components and transfer learning on new systems to cut down weeks of tedious measurements and training. As part of our work, we open-source four datasets, and software for measuring datasets and for training our modular reconstruction.

Problem

• Mask通用性
• 无镜头相机
• 图像重建优化

Innovation

• Modular Learning Reconstruction
• Mask Universality
• Training Efficiency Improvement