Virtual IHC Synthesis and Bimodal Fusion for Patch-Level Classification of HER2 IHC Expression in Breast Cancer
Related Experiment Video
Updated: Nov 2, 2026
Building Up a High-throughput Screening Platform to Assess the Heterogeneity of HER2 Gene Amplification in Breast Cancers
Published on: December 5, 2017
Yi Tang1, Keyu Chen2, Xinru Yang1
1The Affiliated Xuzhou Clinical College of Xuzhou Medical University, Xuzhou, Jiangsu, People's Republic of China.
Purpose:
Accurate assessment of the human epidermal growth factor receptor 2 (HER2) biomarker plays a pivotal role in diagnosing breast malignancies and guiding therapeutic selection, yet routine immunohistochemistry (IHC) testing involves high expenses, requires extensive manual effort, and is subject to diagnostic inconsistencies among pathologists. To overcome these limitations, we propose a cascaded two-stage framework for virtual HER2 IHC synthesis and patch-level classification of HER2 IHC expression from routine hematoxylin and eosin (H&E)-stained images. The prediction target comprises the four conventional HER2 IHC expression categories: 0, 1+, 2+, and 3+.
Patients And Methods:
The proposed pipeline operates in two distinct phases. First, a progressive domain knowledge-constrained generative model is developed to synthesize virtual HER2 IHC images from H&E inputs by incorporating structural and staining-aware priors. Second, the generated virtual HER2 images are paired with the corresponding H&E patches and jointly processed by a bimodal fusion network with guided purification and feature recalibration for patch-level classification of HER2 IHC expression. Experiments were conducted on a retrospective breast cancer pathology dataset comprising 48 patients with paired H&E and HER2 IHC whole-slide images (WSIs) digitized at 20x magnification, sourced from patients identified at Xuzhou Central Hospital between January 2018 and December 2023, yielding a total of 9,287 paired patches for model training and testing.
Results:
The introduced generative framework attains the lowest Fréchet inception distance (FID) of 22.67 among competing methods, indicating superior pathological realism in the synthesized images. In addition, in the four-class patch-level HER2 classification task, the proposed bimodal framework outperforms unimodal H&E-based baselines, yielding an accuracy of 67.12%, an F1-score of 63.63%, and an area under the curve (AUC) of 84.99%.
Conclusion:
These results demonstrate that the virtual HER2 modality provides complementary HER2-oriented cues beyond native morphology alone and that the proposed framework shows potential as a computational tool for low-cost HER2 assessment from routine H&E images.
