Visual-linguistic Diagnostic Semantic Enhancement for medical report generation

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Natural Language Processing

Background:

• Generative models aid medical report generation from images, but struggle with accurate lesion detection and precise clinical terminology.
• Existing methods often fail to capture subtle lesion details or use inconsistent diagnostic language.

Purpose of the Study:

• To develop an advanced model for generating high-quality medical reports that accurately describe lesion areas using precise clinical terms.
• To bridge the semantic gap between visual features in medical images and linguistic descriptions in reports.

Main Methods:

• Proposed a Visual-Linguistic Diagnostic Semantic Enhancement (VLDSE) model.
• Employed supervised contrastive learning in the Image and Report Semantic Consistency (IRSC) module.
• Introduced Visual Semantic Qualification and Quantification (VSQQ) and Post-hoc Semantic Correction (PSC) modules for enhanced semantics.

Main Results:

• The VLDSE model demonstrated significant improvements on the IU X-RAY and MIMIC-MV datasets.
• Achieved a BLEU-4 score of 18.6% on IU X-RAY, a 12.7% improvement over the baseline.
• Improved BLEU-1 score by 10.7% on the MIMIC-MV dataset.

Conclusions:

• The VLDSE model effectively generates accurate and fluent medical reports, particularly in describing lesion areas.
• The proposed modules enhance the model's ability to interpret visual information and utilize precise clinical language.
• This approach holds promise for improving diagnostic accuracy and efficiency in medical imaging.