Patient-specific multimodal learning with multi-view contrastive alignment for chest X-ray report generation

Area of Science:

• Medical Imaging Analysis
• Artificial Intelligence in Healthcare
• Natural Language Generation

Background:

• Manual radiology report generation is time-consuming for radiologists.
• Current automatic methods often use single X-ray views and lack patient context, limiting accuracy.
• There is a need for advanced systems to improve efficiency and diagnostic precision in radiology.

Purpose of the Study:

• To develop an advanced framework for automatic radiology report generation.
• To improve the accuracy and coherence of reports by incorporating multi-view imaging and patient-specific knowledge.
• To establish new benchmarks for multi-view chest X-ray report generation.

Main Methods:

• Proposed EVOKE framework utilizing multi-view contrastive learning for improved visual representation.
• Integrated a knowledge-guided report generation module using patient indication (symptoms).
• Developed and utilized the Multiview CXR and Two-view CXR datasets for research.

Main Results:

• EVOKE demonstrated superior performance over state-of-the-art methods on multiple datasets.
• Achieved significant improvements in metrics such as F1 RadGraph, BLEU, and F1 CheXbert.
• Validated the effectiveness of multi-view learning and patient knowledge integration.

Conclusions:

• EVOKE represents a significant advancement in automated radiology report generation.
• The framework effectively leverages multi-view imaging and patient data for enhanced diagnostic accuracy.
• The developed datasets and framework will facilitate future research in this domain.