Nucleus subtype classification using inter-modality learning

Area of Science:

• Digital pathology
• Computational biology
• Histology imaging analysis

Background:

• Cellular communication and spatial relationships are crucial for human physiology.
• Hematoxylin and eosin (H&E) staining is a common method in clinical and research settings.
• Current AI models like the Colon Nucleus Identification and Classification (CoNIC) Challenge can only label a limited number of cell types on H&E stains.

Purpose of the Study:

• To develop a novel method for labeling previously un-labelable cell types on virtual H&E images.
• To leverage inter-modality learning by combining multiplexed immunofluorescence (MxIF) data with H&E.
• To enhance the granularity of cell type classification in digital pathology.

Main Methods:

• Utilized multiplexed immunofluorescence (MxIF) histology imaging to identify 14 distinct cell subclasses.
• Employed style transfer techniques to generate virtual H&E images from MxIF data.
• Transferred detailed cell labels from MxIF to the synthesized virtual H&E images for analysis.

Main Results:

• Successfully identified helper T and progenitor cell nuclei on virtual H&E images.
• Achieved positive predictive values of 0.34 ± 0.15 for helper T cells and 0.47 ± 0.1 for progenitor cells.
• Demonstrated the feasibility of transferring high-density labels from MxIF to virtual H&E using inter-modality learning.

Conclusions:

• Inter-modality learning enables the annotation of a wider range of cell types on virtual H&E images.
• This approach significantly expands the potential of AI in automating cell classification for digital pathology.
• The findings represent a promising advancement for detailed cellular analysis in histopathology.