Integrative multimodal hybrid data fusion for mortality prediction

Area of Science:

• Multimodal Machine Learning (MML)
• Deep Learning in Healthcare
• Clinical Informatics

Background:

• Healthcare generates diverse data modalities (tabular EHR, ECG, clinical notes).
• Integrating these modalities presents challenges in data representation, alignment, and fusion strategies.
• Accurate mortality prediction is crucial for timely clinical intervention.

Purpose of the Study:

• To propose and evaluate a novel MML approach for healthcare mortality prediction.
• To fuse tabular data, ECG, and textual doctor's notes for enhanced predictive performance.
• To address MML challenges including data preprocessing, alignment, and fusion strategy selection.

Main Methods:

• Utilized MIMIC-IV, MIMIC-ECG, and MIMIC-IV-Note datasets.
• Implemented data preprocessing for noise, outliers, and missing values.
• Compared early, late, and hybrid fusion strategies with novel deep learning models incorporating attention mechanisms.

Main Results:

• The proposed MML model achieved a significant increase in performance with an AUC of 0.96.
• Outperformed previous single-modality models in mortality prediction.
• Demonstrated the benefit of a holistic patient view from multimodal data.

Conclusions:

• Multimodal data fusion significantly enhances mortality prediction accuracy in healthcare.
• The developed MML approach offers a holistic patient view, aiding clinical decision-making.
• Future work should address data biases and model interpretability for clinical adoption.