Neural Field-Based 3D Surface Reconstruction of Microstructures from Multi-Detector Signals in Scanning Electron Microscopy

2025-08-05

AI Summary

In scanning electron microscopy (SEM)–based 3D reconstruction from multi-view, multi-detector 2D images, key bottlenecks include discretization-induced surface distortion, reliance on calibration samples, and shadow-induced gradient errors. To address these, we propose NFH-SEM: an end-to-end self-calibrating neural field framework. NFH-SEM eliminates the need for physical calibration targets; instead, it implicitly disentangles shadow effects during training and jointly leverages geometric and photometric cues to enable continuous, differentiable 3D surface representation. By integrating multi-detector signals and employing self-supervised optimization, the method achieves high-fidelity reconstructions on both real datasets (two-photon lithography structures, peach tree pollen, silicon carbide particles) and synthetic data. Quantitative and qualitative evaluations demonstrate substantial improvements in geometric accuracy and generalization capability for complex micro- and nano-scale topographies.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The scanning electron microscope (SEM) is a widely used imaging device in scientific research and industrial applications. Conventional two-dimensional (2D) SEM images do not directly reveal the three-dimensional (3D) topography of micro samples, motivating the development of SEM 3D surface reconstruction methods. However, reconstruction of complex microstructures remains challenging for existing methods due to the limitations of discrete 3D representations, the need for calibration with reference samples, and shadow-induced gradient errors. Here, we introduce NFH-SEM, a neural field-based hybrid SEM 3D reconstruction method that takes multi-view, multi-detector 2D SEM images as input and fuses geometric and photometric information into a continuous neural field representation. NFH-SEM eliminates the manual calibration procedures through end-to-end self-calibration and automatically disentangles shadows from SEM images during training, enabling accurate reconstruction of intricate microstructures. We validate the effectiveness of NFH-SEM on real and simulated datasets. Our experiments show high-fidelity reconstructions of diverse, challenging samples, including two-photon lithography microstructures, peach pollen, and silicon carbide particle surfaces, demonstrating precise detail and broad applicability.

Problem

• Reconstructing 3D microstructures from 2D SEM images
• Overcoming shadow and calibration issues in SEM reconstruction
• Accurately modeling complex surfaces without manual calibration

Innovation

• Neural field-based hybrid 3D reconstruction
• End-to-end self-calibration eliminates manual steps
• Automatic shadow disentanglement during training