Cross-Domain Image Synthesis: Generating H&E from Multiplex Biomarker Imaging

2025-08-05

AI Summary

This study addresses the challenge that multiplex immunofluorescence (mIF) images lack standardized hematoxylin and eosin (H&E)-like morphological context, hindering computer-aided diagnosis (CAD). To bridge this gap, we propose a multi-level vector-quantized generative adversarial network (VQGAN) architecture for high-fidelity cross-domain synthesis of virtual H&E images from mIF inputs. Compared to conventional conditional GANs (cGANs), our method significantly improves semantic consistency and functional utility of generated images. On a public colorectal cancer dataset, the synthesized virtual H&E images achieve superior nuclear segmentation accuracy and tissue classification consistency relative to real H&E, while also outperforming baselines in visual quality and downstream analytical performance. This work represents the first application of hierarchical VQGAN to virtual histological staining, establishing an interpretable and quantitatively evaluable paradigm for morphology–molecular integration in multimodal digital pathology.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Web data generation and simulation
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

While multiplex immunofluorescence (mIF) imaging provides deep, spatially-resolved molecular data, integrating this information with the morphological standard of Hematoxylin & Eosin (H&E) can be very important for obtaining complementary information about the underlying tissue. Generating a virtual H&E stain from mIF data offers a powerful solution, providing immediate morphological context. Crucially, this approach enables the application of the vast ecosystem of H&E-based computer-aided diagnosis (CAD) tools to analyze rich molecular data, bridging the gap between molecular and morphological analysis. In this work, we investigate the use of a multi-level Vector-Quantized Generative Adversarial Network (VQGAN) to create high-fidelity virtual H&E stains from mIF images. We rigorously evaluated our VQGAN against a standard conditional GAN (cGAN) baseline on two publicly available colorectal cancer datasets, assessing performance on both image similarity and functional utility for downstream analysis. Our results show that while both architectures produce visually plausible images, the virtual stains generated by our VQGAN provide a more effective substrate for computer-aided diagnosis. Specifically, downstream nuclei segmentation and semantic preservation in tissue classification tasks performed on VQGAN-generated images demonstrate superior performance and agreement with ground-truth analysis compared to those from the cGAN. This work establishes that a multi-level VQGAN is a robust and superior architecture for generating scientifically useful virtual stains, offering a viable pathway to integrate the rich molecular data of mIF into established and powerful H&E-based analytical workflows.

Problem

• Generating virtual H&E stains from mIF imaging data
• Bridging molecular and morphological analysis via synthetic H&E
• Enhancing CAD tool compatibility with multiplex biomarker data

Innovation

• VQGAN generates virtual H&E from mIF
• Multi-level VQGAN outperforms cGAN baseline
• Virtual stains enable CAD tool integration