Segmentation-before-Staining Improves Structural Fidelity in Virtual IHC-to-Multiplex IF Translation

2026-03-17

AI Summary

This work addresses the distortion of nuclear count, morphology, and spatial arrangement in virtual staining when translating immunohistochemistry (IHC) images to multiplex immunofluorescence (mIF), which compromises the accuracy of clinical metrics such as the Ki67 index. To mitigate this, the authors propose an unsupervised, architecture-agnostic conditional guidance strategy that leverages continuous nuclear probability maps—generated by a pretrained foundation model for nuclei segmentation—as soft priors. This approach is complemented by a variance-preserving regularizer based on local intensity statistics to maintain cellular heterogeneity and structural fidelity in synthesized images. By incorporating continuous probability maps instead of binary masks, the method avoids boundary information loss and seamlessly integrates with diverse generative architectures—including Pix2Pix, U-Net, and diffusion models—without requiring task-specific tuning or additional annotations. Experiments on two independent datasets demonstrate significant improvements in both nuclear counting accuracy and perceptual quality of generated mIF images.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Multiplex immunofluorescence (mIF) enables simultaneous single-cell quantification of multiple biomarkers within intact tissue architecture, yet its high reagent cost, multi-round staining protocols, and need for specialized imaging platforms limit routine clinical adoption. Virtual staining can synthesize mIF channels from widely available brightfield immunohistochemistry (IHC), but current translators optimize pixel-level fidelity without explicitly constraining nuclear morphology. In pathology, this gap is clinically consequential: subtle distortions in nuclei count, shape, or spatial arrangement propagate directly to quantification endpoints such as the Ki67 proliferation index, where errors of a few percent can shift treatment-relevant risk categories. This work introduces a supervision-free, architecture-agnostic conditioning strategy that injects a continuous cell probability map from a pretrained nuclei segmentation foundation model as an explicit input prior, together with a variance-preserving regularization term that matches local intensity statistics to maintain cell-level heterogeneity in synthesized fluorescence channels. The soft prior retains gradient-level boundary information lost by binary thresholding, providing a richer conditioning signal without task-specific tuning. Controlled experiments across Pix2Pix with U-Net and ResNet generators, deterministic regression U-Net, and conditional diffusion on two independent datasets demonstrate consistent improvements in nuclei count fidelity and perceptual quality, as the sole modifications. Code will be made publicly available upon acceptance.

Problem

• virtual staining
• nuclear morphology
• multiplex immunofluorescence
• structural fidelity
• IHC-to-IF translation

Innovation

• virtual staining
• nuclei segmentation prior
• multiplex immunofluorescence
• variance-preserving regularization
• foundation model