Related Experiment Video
Updated: Aug 6, 2026
Visualization, Quantification, and Mapping of Immune Cell Populations in the Tumor Microenvironment
Published on: March 25, 2020
H&E-Referenced Multiplex Immunofluorescence Interpretation in TMA Cores: Spatial Co-localization, Cell Feature
Jun Jiang1,2, Raymond M Moore1, Brenna C Novotny1
1Department of Quantitative Health Sciences, Mayo Clinic, Rochester MN. USA.
Insights
We developed a framework to align multiplexed immunofluorescence (MxIF) and hematoxylin and eosin (H&E) images, enabling better interpretation of the tumor immune microenvironment (TIME) and virtual H&E generation.
Area of Science:
- Histopathology
- Computational Pathology
- Immunohistochemistry
Background:
- Multiplexed immunofluorescence (MxIF) allows detailed tumor immune microenvironment (TIME) analysis but faces challenges with signal degradation.
- Pathologists traditionally use hematoxylin and eosin (H&E) staining for morphology and cross-reference it with MxIF.
Purpose of the Study:
- To develop a framework for aligning H&E and MxIF images for cross-modal analysis.
- To enable quantitative assessment of feature concordance and virtual H&E generation.
Main Methods:
- Image alignment using cell nucleus detections as anchor points via Coherent Point Drift (CPD) and graph-matching refinement.
- Evaluation on ovarian tissue microarrays (TMAs) with restained and serial sections.
Main Results:
- Consistent alignment performance achieved for both restained and serial sections.
- Aligned images facilitated quantitative cross-modal feature assessment.
- Generated virtual H&E images showed comparable cell populations to real H&E.
Conclusions:
- Cell-centric alignment supports integrative multimodal histopathology analysis.
- Enables spatial co-localization, cell feature validation, and virtual H&E generation for TMA cores.
Background:
Multiplexed immunofluorescence (MxIF) enables high-dimensional immune cell phenotyping and detailed characterization of the tumor immune microenvironment (TIME), but complex cyclic staining can introduce signal degradation and uncertainty in cell-level measurements. In routine practice, pathologists rely on hematoxylin and eosin (H&E) staining as the primary reference for tissue morphology and frequently cross-reference H&E to interpret MxIF findings.
Methods:
We present a framework for aligning H&E and MxIF images at the tissue microarray (TMA) core level to support cross-modal analysis. Cell nucleus detections from each modality are used as anchor points, formulating the task as a point-set alignment problem. Coherent Point Drift (CPD) is applied for global alignment, followed by graph-matching-based refinement to improve local consistency. Evaluation on ovarian TMAs demonstrates consistent alignment performance for both restained and serial sections. We further explore the use of restained H&E as a reference for MxIF interpretation and investigate virtual H&E generation from MxIF data as a complementary approach when restained H&E is unavailable.
Results:
The aligned images enable quantitative assessment of cross-modal feature concordance between MxIF-derived measurements and H&E morphology. The generated virtual H&E demonstrated similar cell population compared to the real H&E.
Conclusions:
Cell-centric alignment can facilitate integrative multimodal histopathology analysis for TMA cores in spatial co-localization, cell feature validation, and virtual H&E generation.
