Dual-Control Frequency-Aware Diffusion Model for Depth-Dependent Optical Microrobot Microscopy Image Generation

2026-04-13

AI Summary

This work addresses the limitations of existing optical microrobotics in 3D perception, which stem from the scarcity of large-scale, high-quality microscopic image datasets and the inability of generative models to accurately reproduce depth-dependent optical effects such as diffraction and defocus. To overcome these challenges, we propose Du-FreqNet, a dual-control, frequency-aware diffusion model that employs two independent ControlNet branches to encode 3D point clouds and depth-related mesh layers separately. An adaptive frequency-domain loss function dynamically weights high- and low-frequency components according to focal-plane distance, while differentiable FFT-based supervision enforces physical consistency. Evaluated under a small-sample setting with only 80 images per pose, our method achieves a 20.7% improvement in SSIM over baseline approaches, significantly enhancing generalization to unseen poses and boosting downstream performance in 3D pose and depth estimation.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Optical microrobots actuated by optical tweezers (OT) are important for cell manipulation and microscale assembly, but their autonomous operation depends on accurate 3D perception. Developing such perception systems is challenging because large-scale, high-quality microscopy datasets are scarce, owing to complex fabrication processes and labor-intensive annotation. Although generative AI offers a promising route for data augmentation, existing generative adversarial network (GAN)-based methods struggle to reproduce key optical characteristics, particularly depth-dependent diffraction and defocus effects. To address this limitation, we propose Du-FreqNet, a dual-control, frequency-aware diffusion model for physically consistent microscopy image synthesis. The framework features two independent ControlNet branches to encode microrobot 3D point clouds and depth-specific mesh layers, respectively. We introduce an adaptive frequency-domain loss that dynamically reweights high- and low-frequency components based on the distance to the focal plane. By leveraging differentiable FFT-based supervision, Du-FreqNet captures physically meaningful frequency distributions often missed by pixel-space methods. Trained on a limited dataset (e.g., 80 images per pose), our model achieves controllable, depth-dependent image synthesis, improving SSIM by 20.7% over baselines. Extensive experiments demonstrate that Du-FreqNet generalizes effectively to unseen poses and significantly enhances downstream tasks, including 3D pose and depth estimation, thereby facilitating robust closed-loop control in microrobotic systems.

Problem

• optical microrobot
• depth-dependent imaging
• microscopy image generation
• diffraction effects
• data scarcity

Innovation

• frequency-aware diffusion model
• dual-control architecture
• adaptive frequency-domain loss
• depth-dependent microscopy synthesis
• differentiable FFT supervision