A Multilevel Transfer Learning Technique and LSTM Framework for Generating Medical Captions for Limited CT and DBT...

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Computer Vision

Background:

• Medical image captioning is crucial but challenging, especially for images with multiple organs.
• Deep learning offers potential for medical image analysis and report generation.
• Limited medical datasets hinder the effectiveness of traditional machine learning approaches.

Purpose of the Study:

• To develop an effective medical image captioning model for scarce datasets.
• To leverage transfer learning to overcome data limitations in medical imaging.
• To enhance the accuracy and detail of generated medical image captions.

Main Methods:

• A Multi-Level Transfer Learning (MLTL) framework was designed with three models, transferring knowledge from non-medical to medical image domains.
• A Long Short-Term-Memory (LSTM) model was employed for sequence generation of captions.
• An enhanced multi-input Convolutional Neural Network (CNN) with feature extraction was used to improve caption precision.

Main Results:

• The proposed model achieved high accuracy (96.90%) and a BLEU score of 76.9%.
• The model demonstrated strong performance despite working with very limited medical datasets.
• Knowledge transfer from non-medical to medical domains significantly improved learning in the target domain.

Conclusions:

• The integrated MLTL and LSTM framework offers a robust solution for medical image captioning with limited data.
• The enhanced CNN model effectively extracts key image features for detailed caption generation.
• This approach addresses the need for advanced automated medical image analysis tools.