Analyzing Information Disparities across Modalities in Mortality Prediction.

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Informatics
• Medical Imaging Analysis

Background:

• Deep learning enables integrating diverse data for clinical prediction using electronic health records (EHRs).
• Chest radiographs (CXRs) offer visual data crucial for predicting intensive care unit (ICU) patient outcomes.
• The comparative value of raw CXRs versus radiology reports for prediction is under-explored.

Purpose of the Study:

• To compare the predictive performance of raw CXRs against radiology reports for 30-day post-discharge mortality in ICU patients.
• To evaluate how different CXR representations augment patient discharge notes for mortality prediction.
• To identify the most informative data modality for enhancing clinical prediction models.

Main Methods:

• Utilized a Vision-Language Model (VLM) incorporating patient discharge notes.
• Compared models augmented with raw CXRs versus radiology reports against a baseline of discharge notes alone.
• Analyzed a subset of the MIMIC-IV dataset (n=1,360) for 30-day mortality prediction.

Main Results:

• Augmenting discharge notes with raw CXRs yielded the highest predictive performance (AUROC = 0.843).
• The CXR-augmented model outperformed the discharge-note-only model (AUROC = 0.816) and the radiology-report-augmented model (AUROC = 0.804).
• Radiology reports were found to omit clinically relevant findings present in CXRs, indicating CXRs contain richer prognostic information.

Conclusions:

• Raw CXRs provide superior prognostic signals for mortality risk prediction compared to radiology reports when augmenting EHR data.
• Selecting the correct data modality is critical for developing effective clinical AI systems.
• Textual summaries like radiology reports may not capture essential predictive information present in raw medical images.