LGD-Net: Latent-Guided Dual-Stream Network for HER2 Scoring with Task-Specific Domain Knowledge

2026-02-19

AI Summary

This work proposes LGD-Net, a novel framework for predicting HER2 status directly from hematoxylin and eosin (H&E)-stained whole-slide images without explicitly generating virtual immunohistochemistry (IHC) images. Addressing the high cost and resource dependency of conventional HER2 IHC testing—and circumventing the computational burden and reconstruction artifacts associated with pixel-level virtual staining—LGD-Net leverages a cross-modal feature hallucination mechanism to map H&E morphological features into the latent space of IHC molecular representations. The architecture integrates teacher-guided distillation, a dual-stream design, and lightweight, domain knowledge–driven auxiliary tasks (e.g., nuclear distribution and membrane staining intensity) to enhance both discriminative power and interpretability. Evaluated on the BCI dataset, the method achieves state-of-the-art HER2 scoring performance using only H&E inputs, significantly outperforming existing baselines.

Technology Category

• Machine Learning: Multi-instance/Multi-view Learning
• Computer Vision: Other Foundations of Computer Vision
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

It is a critical task to evalaute HER2 expression level accurately for breast cancer evaluation and targeted treatment therapy selection. However, the standard multi-step Immunohistochemistry (IHC) staining is resource-intensive, expensive, and time-consuming, which is also often unavailable in many areas. Consequently, predicting HER2 levels directly from H&E slides has emerged as a potential alternative solution. It has been shown to be effective to use virtual IHC images from H&E images for automatic HER2 scoring. However, the pixel-level virtual staining methods are computationally expensive and prone to reconstruction artifacts that can propagate diagnostic errors. To address these limitations, we propose the Latent-Guided Dual-Stream Network (LGD-Net), a novel framework that employes cross-modal feature hallucination instead of explicit pixel-level image generation. LGD-Net learns to map morphological H&E features directly to the molecular latent space, guided by a teacher IHC encoder during training. To ensure the hallucinated features capture clinically relevant phenotypes, we explicitly regularize the model training with task-specific domain knowledge, specifically nuclei distribution and membrane staining intensity, via lightweight auxiliary regularization tasks. Extensive experiments on the public BCI dataset demonstrate that LGD-Net achieves state-of-the-art performance, significantly outperforming baseline methods while enabling efficient inference using single-modality H&E inputs.

Problem

• HER2 scoring
• H&E slides
• virtual IHC
• breast cancer
• domain knowledge

Innovation

• feature hallucination
• latent space mapping
• domain knowledge regularization
• dual-stream network
• virtual IHC