Learning to predict RNA sequence expressions from whole slide images with applications for search and classification

Area of Science:

• Computational pathology
• Bioinformatics
• Artificial intelligence in medicine

Background:

• Deep learning is crucial in digital pathology for diagnosis and prognosis.
• Extracting molecular features from whole slide images (WSIs) is challenging due to cost and tissue requirements.
• Existing methods struggle with integrating morphological and molecular data.

Purpose of the Study:

• To develop an attention-based deep learning model, tRNAsformer, for predicting bulk RNA sequencing (RNA-seq) from WSIs.
• To simultaneously represent WSIs and predict molecular features.
• To address the challenge of limited pixel-level annotations using multiple instance learning.

Main Methods:

• Proposed tRNAsformer, an attention-based deep learning architecture.
• Employed multiple instance learning for weakly supervised learning on WSIs.
• Trained the model to predict bulk RNA-seq from image data.

Main Results:

• Achieved superior performance compared to state-of-the-art algorithms.
• Demonstrated faster convergence during training.
• Successfully integrated tissue morphology with molecular fingerprints from biopsy samples.

Conclusions:

• tRNAsformer effectively predicts molecular features from WSIs, overcoming limitations of traditional molecular tests.
• The model serves as a valuable computational pathology tool for next-generation search and classification.
• Combines morphological and molecular information for enhanced diagnostic and prognostic capabilities.