Development and external validation of deep learning clinical prediction models using variable-length time series...

Area of Science:

• Clinical informatics
• Machine learning in healthcare
• Time series analysis

Background:

• Accurate prediction of clinical outcomes from electronic health records is crucial for patient care.
• Variable-length time series data presents unique challenges for predictive modeling.
• Deep learning architectures offer potential for analyzing complex clinical data.

Purpose of the Study:

• To compare and externally validate deep learning models for variable-length time series data.
• To evaluate different data transformation methods in conjunction with deep learning architectures.
• To assess model performance across three distinct clinical tasks: clinical deterioration, severe acute kidney injury (AKI), and suspected infection.

Main Methods:

• A multicenter retrospective study using data from 2007-2022 across two medical centers.
• Creation of distinct datasets for each clinical task, with one site for training and the other for testing.
• Comparison of three feature engineering methods (normalization, standardization, PLE-DT) and three architectures (LSTM/GRU, TCN, TDW-CNN).
• Evaluation of model discrimination using Area Under the Precision-Recall Curve (AUPRC) and Area Under the Receiver Operating Characteristic Curve (AUROC).

Main Results:

• The study included over 373,000 training admissions and 256,000 testing admissions.
• Long short-term memory/gated recurrent unit (LSTM/GRU) and time-distributed wrapper with convolutional neural network (TDW-CNN) models achieved the highest mean AUPRC in two tasks.
• LSTM/GRU models demonstrated the highest mean AUROC across all tasks (deterioration: 0.81, AKI: 0.92, infection: 0.87).
• The combination of PLE-DT with LSTM/GRU yielded the highest AUPRC across all clinical tasks.

Conclusions:

• The LSTM/GRU architecture with PLE-DT data transformation showed superior external validation performance for predicting clinical outcomes.
• While multiple models performed similarly using AUROC, the LSTM/GRU-PLE-DT approach excelled in AUPRC.
• The LSTM architecture remains competitive with newer models, and PLE-DT can enhance AUPRC for variable-length time series clinical prediction.