Multimodal Deep Learning for Stroke Prediction and Detection using Retinal Imaging and Clinical Data

Area of Science:

• Neuroscience
• Ophthalmology
• Artificial Intelligence

Background:

• Stroke is a significant global health issue.
• Current stroke diagnosis relies on expensive medical imaging.
• Retinal imaging offers a potential cost-effective alternative for assessing cerebrovascular health.

Purpose of the Study:

• To investigate the use of retinal images and clinical data for stroke detection and risk prediction.
• To develop and evaluate a multimodal deep neural network for this purpose.

Main Methods:

• A multimodal deep neural network was developed, processing Optical Coherence Tomography (OCT) and infrared reflectance retinal scans.
• The model integrated clinical data including demographics, vital signs, and diagnosis codes.
• Self-supervised learning was used for pretraining on a large dataset, followed by fine-tuning and evaluation on a labeled subset.

Main Results:

• The multimodal model demonstrated effectiveness in detecting retinal changes associated with acute stroke.
• It accurately predicted future stroke risk within a specified time horizon.
• Achieved a 5% AUROC improvement over unimodal image baselines and an 8% improvement over a state-of-the-art foundation model.

Conclusions:

• Retinal imaging, when combined with clinical data, holds significant potential for identifying high-risk stroke patients.
• The proposed deep learning framework offers a non-invasive and cost-effective method for stroke risk assessment.
• This approach can contribute to early intervention, mitigating the global stroke burden and improving patient outcomes.