Utilizing Longitudinal Chest X-Rays and Reports to Pre-fill Radiology Reports

Area of Science:

• Artificial Intelligence in Medicine
• Medical Imaging Informatics
• Natural Language Processing for Healthcare

Background:

• Speech recognition software reduces radiology reporting times but persistent communication errors remain a challenge.
• Existing methods for generating medical reports often overlook the value of longitudinal patient data.
• Accurate interpretation of radiology reports is crucial for patient care.

Purpose of the Study:

• To address the gap in utilizing longitudinal patient data for radiology report generation.
• To propose a novel approach for pre-filling the "findings" section of radiology reports using multi-modal longitudinal data.
• To mitigate reporting errors by incorporating historical patient information.

Main Methods:

• Developed the "Longitudinal-MIMIC" dataset using 26,625 patients from the MIMIC-CXR dataset.
• Trained a transformer-based model incorporating previous CXR images, current CXR images, and previous reports.
• Employed a cross-attention multi-modal fusion module and a hierarchical memory-driven decoder to process longitudinal information.

Main Results:

• The proposed model significantly outperforms recent approaches in pre-filling radiology report findings.
• Achieved improvements of ≥3% in F1 score and ≥2% in BLEU-4, METEOR, and ROUGE-L metrics.
• Demonstrated the effectiveness of exploiting longitudinal multi-modal data for enhanced report accuracy.

Conclusions:

• Leveraging longitudinal multi-modal data is a promising strategy to reduce errors in radiology reporting.
• The developed transformer-based model effectively utilizes historical patient visit data for report pre-filling.
• This approach has the potential to enhance the reliability and efficiency of radiology reporting systems.