Label correlated contrastive learning for medical report generation

Area of Science:

• Artificial Intelligence
• Medical Informatics
• Computer Vision

Background:

• Automatic medical report generation aids radiologists and reduces errors.
• Current contrastive learning methods lack specificity for multi-disease patient data.
• This limitation hinders the generation of high-quality, specific medical reports.

Purpose of the Study:

• To propose a label-correlated contrastive learning method for improved medical report generation.
• To address the specificity issue in contrastive learning for multi-label medical data.
• To enhance the accuracy and quality of automatically generated medical reports.

Main Methods:

• A refined similarity description matrix was created using multi-label classification similarities.
• Image features and decoder embeddings were projected into a shared hidden space.
• Label-correlated contrastive learning was applied, weighting harder negative samples with shared labels more heavily.
• The method combines label-correlated contrastive learning with an attention mechanism.

Main Results:

• Experiments on IU X-ray and MIMIC-CXR datasets demonstrated superior performance.
• Achieved METEOR (0.198) and ROUGE-L (0.392) on IU X-ray.
• Achieved precision (0.384), recall (0.376), and F-1 (0.304) on MIMIC-CXR.
• Outperformed previous state-of-the-art models in medical report generation.

Conclusions:

• The proposed method significantly improves automatic medical report generation.
• This advancement makes automated reports more feasible for computer-aided diagnosis.
• Enhanced specificity in report generation leads to better clinical utility.