Predicting Semantic Descriptions from Medical Images with Convolutional Neural Networks

Area of Science:

• Medical imaging analysis
• Computational pathology
• Ophthalmology imaging

Background:

• Accurate computational models from medical imaging demand extensive data.
• Voxel-level annotation is often impractical for large datasets.
• Clinical reports contain valuable, underutilized knowledge.

Purpose of the Study:

• To develop a weakly supervised learning approach for medical image analysis.
• To leverage semantic information from clinical reports as a learning target.
• To improve voxel-level classification accuracy in optical coherence tomography (OCT) volumes.

Main Methods:

• Utilized a convolutional neural network to predict semantic representations from imaging data.
• Trained models using volume-level semantic descriptions from clinical reports.
• Applied the method to a dataset of 157 OCT volumes.

Main Results:

• Achieved accurate voxel-level classifiers from weak, volume-level semantic labels.
• Demonstrated improved classification accuracy for intraretinal cystoid fluid (IRC), subretinal fluid (SRF), and normal retinal tissue.
• Showcased the algorithm's ability to link semantic concepts to image content and geometry.

Conclusions:

• Semantic representations of clinical reports can effectively guide weakly supervised learning in medical imaging.
• This approach enhances the utility of routinely generated clinical data for model training.
• The method shows promise for improving the automated analysis of OCT scans.