Comparison of Multimodal Deep Learning Approaches for Predicting Clinical Deterioration in Ward Patients: An...

Area of Science:

• Clinical Informatics
• Machine Learning in Healthcare
• Patient Monitoring

Background:

• Machine learning models for clinical deterioration prediction decrease morbidity and mortality.
• Current models often have high false positive rates and rely solely on structured data.

Purpose of the Study:

• To compare the predictive performance of models using structured data alone versus those incorporating unstructured data from clinical notes.
• To evaluate different methods of parameterizing clinical note information for machine learning models.

Main Methods:

• Adult patients from two large cohorts (University of Chicago and University of Wisconsin-Madison) were analyzed.
• Concept Unique Identifiers (CUIs) from clinical notes were extracted and parameterized using Standard Tokenization, ICD Rollup (Tokenization and Binary Variables), SapBERT Embeddings, and CUI Clustering.
• Deep recurrent neural networks were used to compare models with and without CUIs for predicting intensive care unit transfer or death within 24 hours.

Main Results:

• The SapBERT Embeddings (SE) model showed the highest Area Under the Precision-Recall Curve (AUPRC).
• The CUI Clustering (CC) and structured-only models achieved the highest Area Under the Receiver Operating Characteristic Curve (AUROC).
• Models incorporating CUIs demonstrated comparable or slightly improved performance, with specific CUIs like 'NPO - Nothing by mouth' and 'Chemotherapy' being highly predictive.

Conclusions:

• While multimodal models integrating structured data and SapBERT embeddings showed high AUPRC, overall performance gains from including clinical note CUIs were minimal.
• The addition of CUIs did not substantially enhance prediction accuracy for clinical deterioration.
• Models utilizing CUIs can offer clinicians supplementary information and context to aid decision-making.