ProtoMTG: Prototypical Multi-Task Learning for the Generation of Multiple Stained Immunohistochemical Images.

Area of Science:

• Computational pathology
• Artificial intelligence in medicine
• Biomedical image analysis

Background:

• Multiplex immunohistochemistry (mIHC) enables tumor microenvironment assessment but is costly and time-consuming.
• Machine learning-based virtual staining offers a faster alternative for generating mIHC markers.
• Existing virtual staining models often generate markers independently, limiting interpretability and overlooking inter-marker relationships.

Purpose of the Study:

• To propose an explainable prototypical multi-task generation framework (ProtoMTG) for simultaneous virtual staining of multiple mIHC markers.
• To capture interrelationships among different virtual staining tasks using shared and task-specific prototypes.
• To improve the interpretability and efficiency of virtual staining for mIHC images.

Main Methods:

• Developed ProtoMTG, a novel framework incorporating a multi-task prototype layer and a proto-attention layer.
• Introduced prototypical activation and diversity losses for enhanced prototype representation.
• Created and utilized three benchmark mIHC datasets from colon, liver, and stomach tissues for evaluation.

Main Results:

• ProtoMTG successfully generated multiple mIHC markers simultaneously, outperforming existing image generation models.
• The framework demonstrated significant explainable ability in the virtual staining of mIHC markers.
• Experimental results validated the effectiveness of ProtoMTG on diverse organ datasets.

Conclusions:

• ProtoMTG provides an efficient and interpretable solution for virtual multiplex immunohistochemistry staining.
• The proposed method advances the application of AI in analyzing complex tumor microenvironments.
• The framework's ability to capture marker interrelationships offers new insights into tissue analysis.