PAINT: Pathology-Aware Integrated Next-Scale Transformation for Virtual Immunohistochemistry

2026-01-22

AI Summary

This work addresses the challenges in synthesizing immunohistochemistry (IHC) stains from routine hematoxylin and eosin (H&E) images, where ambiguous correlations between tissue morphology and protein expression, along with inconsistent structural semantics, hinder reliable generation. To overcome these issues, the authors propose a structure-prioritized visual autoregressive framework that formulates virtual IHC synthesis as a conditional generation task guided by global structural layout. Central to this approach is the introduction of a Spatial Structure Start Map (3S-Map), which enables spatially aligned, deterministic autoregressive initialization to ensure structural and semantic consistency in the generated IHC images. Evaluated on the IHC4BC and MIST datasets, the method significantly outperforms existing approaches, achieving superior performance in both structural fidelity and downstream clinical tasks.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Structured Learning
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Virtual immunohistochemistry (IHC) aims to computationally synthesize molecular staining patterns from routine Hematoxylin and Eosin (H\&E) images, offering a cost-effective and tissue-efficient alternative to traditional physical staining. However, this task is particularly challenging: H\&E morphology provides ambiguous cues about protein expression, and similar tissue structures may correspond to distinct molecular states. Most existing methods focus on direct appearance synthesis to implicitly achieve cross-modal generation, often resulting in semantic inconsistencies due to insufficient structural priors. In this paper, we propose Pathology-Aware Integrated Next-Scale Transformation (PAINT), a visual autoregressive framework that reformulates the synthesis process as a structure-first conditional generation task. Unlike direct image translation, PAINT enforces a causal order by resolving molecular details conditioned on a global structural layout. Central to this approach is the introduction of a Spatial Structural Start Map (3S-Map), which grounds the autoregressive initialization in observed morphology, ensuring deterministic, spatially aligned synthesis. Experiments on the IHC4BC and MIST datasets demonstrate that PAINT outperforms state-of-the-art methods in structural fidelity and clinical downstream tasks, validating the potential of structure-guided autoregressive modeling.

Problem

• Virtual Immunohistochemistry
• H&E to IHC Synthesis
• Cross-modal Generation
• Semantic Consistency
• Molecular Staining

Innovation

• structure-guided generation
• autoregressive modeling
• virtual immunohistochemistry
• spatial structural prior
• cross-modal synthesis