Ultrasound Report Generation With Cross-Modality Feature Alignment via Unsupervised Guidance

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Clinical Informatics

Background:

• Automatic report generation is crucial in computer-aided diagnosis to reduce clinician workload.
• Generating accurate reports from medical images, like ultrasounds, presents challenges in aligning visual and textual data.

Purpose of the Study:

• To propose a novel framework for automatic ultrasound report generation.
• To address the feature discrepancy challenge between ultrasound images and textual reports.
• To enhance the comprehensiveness and accuracy of generated medical reports.

Main Methods:

• Utilized a combination of unsupervised and supervised learning methods.
• Incorporated unsupervised learning to extract knowledge from ultrasound text reports as prior information.
• Designed a global semantic comparison mechanism to improve report generation performance.

Main Results:

• The proposed framework demonstrated superior performance across three large-scale ultrasound image-text datasets.
• Achieved enhanced alignment between visual and textual features in ultrasound reports.
• Outperformed existing state-of-the-art approaches in automatic report generation.

Conclusions:

• The novel framework effectively generates accurate and comprehensive ultrasound reports.
• The integration of unsupervised and supervised learning, along with semantic comparison, significantly improves report generation.
• The developed datasets and framework offer valuable resources for advancing research in medical image report generation.