Abnormality-aware multimodal learning for WSI classification.

Area of Science:

• Computational pathology
• Digital pathology
• Artificial intelligence in oncology

Background:

• Whole slide images (WSIs) are crucial for cancer diagnosis but present computational challenges due to their size and reliance on visual data.
• Current methods struggle with computational demands, focus on relevant regions, and often overlook multimodal data like cellular and textual information.

Purpose of the Study:

• To develop an efficient and accurate computational framework for analyzing WSIs in cancer diagnosis and subtyping.
• To address limitations of existing methods by integrating abnormality detection and multimodal feature learning.

Main Methods:

• Proposed the Abnormality-Aware MultiModal (AAMM) learning framework.
• Utilized a Gaussian Mixture Variational Autoencoder (GMVAE) for identifying informative patches and reducing computational load.
• Integrated multimodal features (patch-level, cell-level, text-level) using pathology-specific foundation models and cross-attention mechanisms.

Main Results:

• The AAMM framework demonstrated superior performance in normal-tumor classification and cancer subtyping compared to state-of-the-art methods.
• Achieved efficient and scalable analysis of WSIs by combining abnormal region detection with multimodal data integration.

Conclusions:

• The AAMM framework offers an effective solution for enhancing computational pathology.
• This approach advances accurate and efficient cancer diagnosis and subtyping using WSIs by leveraging multimodal data and intelligent patch selection.