Seeing Clearly and Deeply: An RGBD Imaging Approach with a Bio-inspired Monocentric Design

2025-10-29

AI Summary

To address the trade-off between RGB image sharpness and depth estimation accuracy in compact RGB-D imaging, this paper proposes a bio-inspired single-center omnidirectional imaging framework. It employs a physically interpretable omnidirectional optical design whose depth-dependent point spread function (PSF) naturally encodes depth information—eliminating the need for diffractive elements or chromatic aberration correction. We establish an optics-algorithm co-design paradigm, integrating a physics-based forward model with an end-to-end differentiable multi-scale dual-head reconstruction network featuring a shared encoder. Trained on synthetic data, our method achieves state-of-the-art depth estimation performance (Abs Rel = 0.026, RMSE = 0.130), significantly outperforming existing monocular approaches. Simultaneously, RGB reconstruction attains SSIM = 0.960 and LPIPS = 0.082, demonstrating the best-known balance between image fidelity and depth accuracy for compact RGB-D systems.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Mixed Discrete/Continuous Search
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Achieving high-fidelity, compact RGBD imaging presents a dual challenge: conventional compact optics struggle with RGB sharpness across the entire depth-of-field, while software-only Monocular Depth Estimation (MDE) is an ill-posed problem reliant on unreliable semantic priors. While deep optics with elements like DOEs can encode depth, they introduce trade-offs in fabrication complexity and chromatic aberrations, compromising simplicity. To address this, we first introduce a novel bio-inspired all-spherical monocentric lens, around which we build the Bionic Monocentric Imaging (BMI) framework, a holistic co-design. This optical design naturally encodes depth into its depth-varying Point Spread Functions (PSFs) without requiring complex diffractive or freeform elements. We establish a rigorous physically-based forward model to generate a synthetic dataset by precisely simulating the optical degradation process. This simulation pipeline is co-designed with a dual-head, multi-scale reconstruction network that employs a shared encoder to jointly recover a high-fidelity All-in-Focus (AiF) image and a precise depth map from a single coded capture. Extensive experiments validate the state-of-the-art performance of the proposed framework. In depth estimation, the method attains an Abs Rel of 0.026 and an RMSE of 0.130, markedly outperforming leading software-only approaches and other deep optics systems. For image restoration, the system achieves an SSIM of 0.960 and a perceptual LPIPS score of 0.082, thereby confirming a superior balance between image fidelity and depth accuracy. This study illustrates that the integration of bio-inspired, fully spherical optics with a joint reconstruction algorithm constitutes an effective strategy for addressing the intrinsic challenges in high-performance compact RGBD imaging. Source code will be publicly available at https://github.com/ZongxiYu-ZJU/BMI.

Problem

• Achieving high-fidelity compact RGBD imaging with accurate depth estimation
• Overcoming limitations of conventional optics and software-only depth estimation methods
• Resolving trade-offs between optical complexity and imaging performance in RGBD systems

Innovation

• Bio-inspired monocentric lens encodes depth in PSFs
• Physically-based forward model generates synthetic dataset
• Dual-head network jointly recovers AiF image and depth map