Related Experiment Video
Updated: Jan 14, 2026
Multiplexed Fluorescent Immunohistochemical Staining, Imaging, and Analysis in Histological Samples of Lymphoma
Published on: January 9, 2019
Virtual multiplex immunofluorescence identifies lymphocyte subsets predictive of response to neoadjuvant therapy
Anran Li1, Madeleine Torcasso1,2, Anna Woodard1
1Department of Medicine, University of Chicago, Chicago, IL, USA.
Insights
A new deep learning tool, mSIGHT, translates standard H&E stains into virtual multiplex immunofluorescence images. This method identifies immune biomarkers predictive of breast cancer treatment response, offering a scalable alternative to complex imaging techniques.
Area of Science:
- Computational pathology
- Artificial intelligence in oncology
- Biomarker discovery
Background:
- Hematoxylin and eosin (H&E) staining is a standard pathology technique but lacks cellular specificity.
- Multiplex immunofluorescence (mIF) provides spatial immune insights but is costly and complex for clinical use.
- Novel methods are needed to extract immune information from routine H&E histology.
Purpose of the Study:
- To develop and validate a deep learning tool for translating H&E images into synthetic mIF images.
- To preserve immune cell information predictive of treatment response in breast cancer.
- To create a cost-effective and scalable alternative to mIF.
Main Methods:
- A novel pipeline, mSIGHT, was developed, integrating a registration network to align H&E and mIF images.
- The pipeline outperformed standard Pix2Pix and CycleGAN models in image translation.
- Generated synthetic mIF images were evaluated using pixel-level and biological metrics, including cell density and proximity.
Main Results:
- mSIGHT successfully generated synthetic mIF images that preserved immune cell distributions and proximity metrics.
- In a cohort of 218 breast cancer cases, predicted CD8+ T cell density was significantly associated with complete response to neoadjuvant chemotherapy.
- This association was independent of established prognostic factors.
Conclusions:
- The mSIGHT pipeline offers a scalable and affordable method to generate virtual mIF images from H&E slides.
- Interpretable immune biomarkers can be derived, aiding in the personalization of neoadjuvant therapy.
- mSIGHT has the potential to enhance treatment selection and monitoring in breast cancer.
Background:
Hematoxylin and eosin (H&E) staining is routine in pathology but lacks cellular specificity. Multiplex immunofluorescence (mIF) captures spatial immune relationships in tumors, but cost and complexity limit clinical application. Novel approaches to yield similar information from readily available tumor histology are needed.
Objectives:
Develop and validate a novel deep learning tool capable of translating standard H&E-stained histopathology images into high-fidelity synthetic mIF images that preserve immune cell information predictive of treatment response in breast cancer.
Design:
Comparative model evaluation and predictive modeling in a retrospective breast cancer cohort.
Methods:
Core-needle biopsies from 17 triple-negative breast cancer cases underwent mIF imaging. Hematoxylin and eosin and mIF images for DAPI (nuclei), pan-CK (tumor), CD3/CD4/CD8 (T-cells), and CD20 (B cells) were aligned. A pipeline outperforming standard Pix2Pix and CycleGAN image translation networks was developed, "multiplex Synthetic Immunofluoresence Generated through H&E Translation" (mSIGHT), which integrates a registration network to overcome misalignment between the input and target images. Generated images were evaluated with pixel-level metrics and biological metrics, including cell density and cell-to-cell adjacency. The pipeline was then applied to an external cohort to assess associations between predicted immune features and pathologic response to neoadjuvant chemotherapy.
Results:
Generated images preserved immune cell distributions and proximity metrics correlated to the ground truth cell counts. In a cohort of 218 breast cancer cases treated with neoadjuvant chemotherapy, predicted density of CD8+ T cells was significantly associated with complete response (adjusted odds ratio 1.89, 95% confidence interval 1.23-2.80, p = 0.002), independent of receptor status, grade, and pathologist TIL annotations.
Conclusion:
The mSIGHT pipeline enables translation of routine H&E slides into virtual mIF images with interpretable immune biomarkers, offering a scalable and affordable alternative to multiplex imaging. It also identifies immune features predictive of therapeutic response and has the potential to assist in the personalization of neoadjuvant therapy.
More Related Videos
09:32Multiplexed Immunofluorescence Analysis and Quantification of Intratumoral PD-1+ Tim-3+ CD8+ T Cells
Published on: February 8, 2018
06:05Author Spotlight: Multiplex Immunofluorescence Combined with Spatial Image Analysis for the Clinical and Biological Assessment of the Tumor Microenvironment
Published on: June 2, 2023