MKCL: Medical Knowledge with Contrastive Learning model for radiology report generation

Area of Science:

• Artificial Intelligence
• Medical Imaging Analysis
• Natural Language Processing

Background:

• Automatic radiology report generation aims to improve diagnostic accuracy and efficiency.
• Current deep learning methods struggle with visual-textual data bias and lack of medical knowledge integration.
• Unlabeled medical images and the interplay of medical findings pose significant challenges.

Purpose of the Study:

• To propose a Medical Knowledge with Contrastive Learning (MKCL) model for enhanced automatic radiology report generation.
• To address data bias and leverage unlabeled medical images for improved accuracy.
• To integrate medical knowledge and visual features for more precise disease finding identification.

Main Methods:

• Developed the MKCL model incorporating an IU Medical Knowledge Graph (IU-MKG) for understanding relationships between medical findings.
• Designed Knowledge Enhanced Attention (KEA) to integrate IU-MKG with visual features, mitigating textual data bias.
• Employed supervised contrastive learning to effectively utilize unlabeled radiographic images for abnormality detection.

Main Results:

• The MKCL model demonstrated superior performance compared to state-of-the-art methods on the IU X-ray dataset.
• Ablation studies confirmed the significant contributions of the IU-MKG and supervised contrastive learning modules.
• The model showed enhanced capabilities in detecting abnormal regions and accurately describing findings.

Conclusions:

• The proposed MKCL model effectively enhances automatic radiology report generation by integrating medical knowledge and contrastive learning.
• KEA and supervised contrastive learning are crucial components for improving accuracy and handling data limitations.
• MKCL offers a promising approach to reduce radiologist workload and improve healthcare efficiency.