MetaScope: Optics-Driven Neural Network for Ultra-Micro Metalens Endoscopy

2025-08-05

AI Summary

Existing micro-endoscopes are constrained by millimeter-thick conventional convex lenses, limiting their utility in microscopic clinical applications. While emerging metalenses offer ultra-thin profiles, their practical deployment is hindered by severe intensity attenuation and chromatic aberration, compounded by the absence of dedicated datasets and algorithms. To address this, we propose MetaScope—a novel neural framework featuring optical-perception-driven, intensity-adaptive adjustment and chromatic aberration correction modules, integrated with gradient-guided knowledge distillation for physics-informed end-to-end image restoration and segmentation. The architecture unifies metasurface optical simulation, point-spread-function (PSF) modeling, optical embedding learning, and data-physics co-optimization. On metalens-based image restoration and segmentation benchmarks, MetaScope surpasses state-of-the-art methods. Moreover, it demonstrates strong generalizability and clinical applicability in real biological tissue imaging.

Technology Category

• Computer Vision: Segmentation
• Search and Optimization: Metareasoning and Metaheuristics
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Security and Privacy: Browser and end-point security
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Miniaturized endoscopy has advanced accurate visual perception within the human body. Prevailing research remains limited to conventional cameras employing convex lenses, where the physical constraints with millimetre-scale thickness impose serious impediments on the micro-level clinical. Recently, with the emergence of meta-optics, ultra-micro imaging based on metalenses (micron-scale) has garnered great attention, serving as a promising solution. However, due to the physical difference of metalens, there is a large gap in data acquisition and algorithm research. In light of this, we aim to bridge this unexplored gap, advancing the novel metalens endoscopy. First, we establish datasets for metalens endoscopy and conduct preliminary optical simulation, identifying two derived optical issues that physically adhere to strong optical priors. Second, we propose MetaScope, a novel optics-driven neural network tailored for metalens endoscopy driven by physical optics. MetaScope comprises two novel designs: Optics-informed Intensity Adjustment (OIA), rectifying intensity decay by learning optical embeddings, and Optics-informed Chromatic Correction (OCC), mitigating chromatic aberration by learning spatial deformations informed by learned Point Spread Function (PSF) distributions. To enhance joint learning, we further deploy a gradient-guided distillation to transfer knowledge from the foundational model adaptively. Extensive experiments demonstrate that MetaScope not only outperforms state-of-the-art methods in both metalens segmentation and restoration but also achieves impressive generalized ability in real biomedical scenes.

Problem

• Bridging data-algorithm gap in ultra-micro metalens endoscopy
• Addressing intensity decay and chromatic aberration in metalens imaging
• Enhancing metalens endoscopy performance via optics-driven neural network

Innovation

• Optics-driven neural network for metalens endoscopy
• Optics-informed Intensity Adjustment rectifies decay
• Optics-informed Chromatic Correction mitigates aberration