Learned split-spectrum metalens for obstruction-free broadband imaging in the visible

2026-01-27

AI Summary

This work addresses the degradation of visible broadband imaging quality in compact optical systems caused by near-field occluders such as raindrops, dust, or fences. The authors propose a learning-based spectral-splitting metalens that jointly designs multi-band spectral filtering and metalens optics for the first time. By partitioning the RGB channels into passbands and stopbands, the system focuses distant scenes in the passbands while suppressing near-field occluder contributions in the stopbands. An end-to-end neural network is integrated for image reconstruction. Compared to conventional hyperbolic metalenses, the proposed method achieves a 32.29% improvement in PSNR, a 13.54% gain in object detection mAP, and enhancements of 48.45% and 20.35% in semantic segmentation IoU and mIoU, respectively, effectively overcoming the longstanding trade-off between broadband imaging performance and occlusion robustness.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Obstructions such as raindrops, fences, or dust degrade captured images, especially when mechanical cleaning is infeasible. Conventional solutions to obstructions rely on a bulky compound optics array or computational inpainting, which compromise compactness or fidelity. Metalenses composed of subwavelength meta-atoms promise compact imaging, but simultaneous achievement of broadband and obstruction-free imaging remains a challenge, since a metalens that images distant scenes across a broadband spectrum cannot properly defocus near-depth occlusions. Here, we introduce a learned split-spectrum metalens that enables broadband obstruction-free imaging. Our approach divides the spectrum of each RGB channel into pass and stop bands with multi-band spectral filtering and learns the metalens to focus light from far objects through pass bands, while filtering focused near-depth light through stop bands. This optical signal is further enhanced using a neural network. Our learned split-spectrum metalens achieves broadband and obstruction-free imaging with relative PSNR gains of 32.29% and improves object detection and semantic segmentation accuracies with absolute gains of +13.54% mAP, +48.45% IoU, and +20.35% mIoU over a conventional hyperbolic design. This promises robust obstruction-free sensing and vision for space-constrained systems, such as mobile robots, drones, and endoscopes.

Problem

• obstruction-free imaging
• broadband imaging
• metalens
• visible spectrum
• compact optics

Innovation

• metalens
• split-spectrum
• obstruction-free imaging
• computational imaging
• neural network