A multimodal whole-slide foundation model for pathology.

Area of Science:

• Computational pathology
• Artificial intelligence in medicine
• Foundation models

Background:

• Foundation models advance computational pathology by encoding histopathology regions (ROIs) into transferable features via self-supervised learning.
• Clinical application is limited by scarce data in disease-specific cohorts, particularly for rare conditions.

Purpose of the Study:

• To develop a multimodal whole-slide foundation model for pathology that overcomes data limitations.
• To enable general-purpose slide representation extraction and pathology report generation for diverse clinical tasks.

Main Methods:

• Developed TITAN (Transformer-based pathology Image and Text Alignment Network), a multimodal whole-slide foundation model.
• Pretrained on 335,645 whole-slide images using visual self-supervised learning and vision-language alignment.
• Utilized pathology reports and 423,122 synthetic captions from a generative AI copilot for pathology.

Main Results:

• TITAN extracts general-purpose slide representations and generates pathology reports without fine-tuning or clinical labels.
• The model demonstrates strong generalization in resource-limited scenarios like rare disease retrieval and cancer prognosis.
• TITAN outperforms existing ROI and slide foundation models across various machine learning tasks, including zero-shot classification and cross-modal retrieval.

Conclusions:

• TITAN represents a significant advancement in multimodal foundation models for computational pathology.
• The model's ability to generalize to rare diseases and limited data scenarios holds promise for clinical applications.
• TITAN enhances capabilities in rare cancer retrieval, cross-modal retrieval, and automated pathology report generation.