Towards H&E Referenced Multiplex Immunofluorescence Interpretation: Spatial Co-localization, Cell Feature Validation,...

Area of Science:

• Computational pathology
• Biomedical imaging analysis
• Tumor microenvironment research

Background:

• Multiplexed Immunofluorescence (MxIF) provides detailed immune cell phenotyping within the tumor immune microenvironment (TIME).
• MxIF staining complexity can compromise signal integrity.
• Hematoxylin and Eosin (H&E) staining offers complementary morphological data and is often cross-referenced with MxIF.

Purpose of the Study:

• To develop a novel framework for aligning H&E and MxIF images for precise cross-modal cell feature validation.
• To enable reliable validation of cell-level features across different imaging modalities.
• To explore virtual H&E image generation from MxIF data.

Main Methods:

• Formulated multimodal image registration as a point set alignment problem using cell detection outputs as anchors.
• Employed Coherent Point Drift (CPD) for initial image alignment.
• Utilized Graph Matching (GM) for refinement of the alignment.

Main Results:

• Achieved high alignment accuracy on ovarian cancer tissue microarrays (TMAs).
• Enabled reliable validation of cell-level features across modalities for both restained and serial sections.
• Demonstrated that restained H&E enhances confidence in MxIF findings.
• Showcased the feasibility of generating virtual H&E images from MxIF data.

Conclusions:

• The proposed framework enables precise H&E and MxIF image alignment for robust cross-modal cell feature validation.
• Restained H&E improves confidence in MxIF-derived findings.
• Virtual H&E generation from MxIF offers a viable alternative for integrated multimodal analysis when restained H&E is unavailable.