V2V3D: View-to-View Denoised 3D Reconstruction for Light-Field Microscopy

2025-04-10

AI Summary

Light-field microscopy (LFM) 3D fluorescence reconstruction suffers from high sensitivity to sensor noise and heavy reliance on ground-truth annotations. To address these limitations, we propose the first unsupervised view-to-view joint optimization framework. Our method establishes a self-supervised view-to-view paradigm grounded in the independence assumption of sensor noise; introduces a wave-optics-driven feature alignment mechanism, modeling the physical point spread function as a learnable convolutional kernel to recover high-frequency details; and integrates Noise2Noise principles with end-to-end inter-view consistency constraints. Evaluated on a custom-built LFM dataset, our approach significantly outperforms state-of-the-art methods, achieving simultaneous improvements in reconstruction accuracy, denoising robustness, and inference efficiency—without requiring any ground-truth labels.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Machine Learning: Multi-instance/Multi-view Learning
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Light field microscopy (LFM) has gained significant attention due to its ability to capture snapshot-based, large-scale 3D fluorescence images. However, existing LFM reconstruction algorithms are highly sensitive to sensor noise or require hard-to-get ground-truth annotated data for training. To address these challenges, this paper introduces V2V3D, an unsupervised view2view-based framework that establishes a new paradigm for joint optimization of image denoising and 3D reconstruction in a unified architecture. We assume that the LF images are derived from a consistent 3D signal, with the noise in each view being independent. This enables V2V3D to incorporate the principle of noise2noise for effective denoising. To enhance the recovery of high-frequency details, we propose a novel wave-optics-based feature alignment technique, which transforms the point spread function, used for forward propagation in wave optics, into convolution kernels specifically designed for feature alignment. Moreover, we introduce an LFM dataset containing LF images and their corresponding 3D intensity volumes. Extensive experiments demonstrate that our approach achieves high computational efficiency and outperforms the other state-of-the-art methods. These advancements position V2V3D as a promising solution for 3D imaging under challenging conditions.

Problem

• Unsupervised joint denoising and 3D reconstruction for light-field microscopy
• Enhancing high-frequency detail recovery with wave-optics feature alignment
• Addressing sensitivity to noise and lack of annotated training data

Innovation

• Unsupervised view-to-view denoising and 3D reconstruction
• Wave-optics-based feature alignment technique
• LFM dataset with LF images and 3D volumes