A Curriculum Learning Based Approach to Captioning Ultrasound Images

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Natural Language Processing

Background:

• Medical datasets for image captioning are often small compared to natural image datasets.
• Training effective natural language processing (NLP) models for medical imaging requires specialized approaches.

Purpose of the Study:

• To develop a novel curriculum learning approach for training NLP-based fetal ultrasound image captioning models.
• To address the challenge of limited data in medical image captioning.

Main Methods:

• Proposed a "dual-curriculum" method utilizing both image and text information curricula.
• Compared distance measures for curriculum creation, identifying Wasserstein distance for images and tf-idf for text as optimal.
• Trained models using real-world ultrasound videos with synchronized sonographer speech recordings.

Main Results:

• Curriculum learning significantly improved performance metrics compared to standard mini-batch training for image classification.
• The dual-curriculum approach demonstrated enhanced performance for ultrasound image captioning.
• Wasserstein distance and tf-idf metric proved effective for dual curriculum construction.

Conclusions:

• Curriculum learning is a viable and effective strategy for training NLP models on small medical datasets.
• The proposed dual-curriculum method offers a significant advancement in fetal ultrasound image captioning.
• This approach holds promise for improving diagnostic capabilities through automated image analysis.