Fidelity-preserving enhancement of ptychography with foundational text-to-image models

2025-09-02

AI Summary

Ptychographic phase retrieval suffers from artifacts such as grid ill-posedness and interlayer crosstalk, degrading reconstruction fidelity. To address this, we propose a physics-informed plug-and-play framework that—uniquely for this task—integrates a pre-trained text-to-image diffusion model (LEDITS++) to localize and edit artifact regions via natural language instructions. Embedded within an alternating direction method of multipliers (ADMM) optimization scheme, our approach jointly enforces strict data consistency with measured diffraction patterns while leveraging semantic guidance for physically grounded image enhancement. Evaluated on both simulated and experimental datasets, the method significantly suppresses artifacts, improves structural fidelity and peak signal-to-noise ratio (PSNR), and maintains high diffraction pattern consistency—demonstrating robust physical plausibility and quantitative superiority over conventional approaches.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Learning to Search
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Security and Privacy: Data transparency and provenance

Abstract

Ptychographic phase retrieval enables high-resolution imaging of complex samples but often suffers from artifacts such as grid pathology and multislice crosstalk, which degrade reconstructed images. We propose a plug-and-play (PnP) framework that integrates physics model-based phase retrieval with text-guided image editing using foundational diffusion models. By employing the alternating direction method of multipliers (ADMM), our approach ensures consensus between data fidelity and artifact removal subproblems, maintaining physics consistency while enhancing image quality. Artifact removal is achieved using a text-guided diffusion image editing method (LEDITS++) with a pre-trained foundational diffusion model, allowing users to specify artifacts for removal in natural language. Demonstrations on simulated and experimental datasets show significant improvements in artifact suppression and structural fidelity, validated by metrics such as peak signal-to-noise ratio (PSNR) and diffraction pattern consistency. This work highlights the combination of text-guided generative models and model-based phase retrieval algorithms as a transferable and fidelity-preserving method for high-quality diffraction imaging.

Problem

• Reduces grid pathology and multislice crosstalk artifacts
• Integrates physics-based phase retrieval with text-guided editing
• Ensures data fidelity while removing specified artifacts

Innovation

• Plug-and-play physics-diffusion integration framework
• Text-guided artifact removal via LEDITS++
• ADMM-consensus data fidelity enhancement