CPAE: Contrastive predictive autoencoder for unsupervised pre-training in health status prediction

Area of Science:

• Medical Informatics
• Machine Learning
• Data Science

Background:

• Fully-supervised learning in Electronic Health Records (EHRs) requires substantial labeled data, which is often scarce.
• Leveraging unlabeled EHR data through pre-training is crucial for developing robust health status prediction models.

Purpose of the Study:

• To propose a novel, data-efficient framework called Contrastive Predictive Autoencoder (CPAE) for EHR-based health status prediction.
• To enhance the utilization of unlabeled EHR data through contrastive pre-training followed by task-specific fine-tuning.

Main Methods:

• Developed CPAE, integrating contrastive predictive coding (CPC) for global feature extraction and a reconstruction process for local feature capture.
• Introduced an attention mechanism variant (AtCPAE) to balance global and local feature learning.
• Pre-trained the model on unlabeled EHR data before fine-tuning on downstream prediction tasks.

Main Results:

• CPAE demonstrated superior performance on in-hospital mortality and length-of-stay prediction tasks compared to supervised methods and CPC.
• The AtCPAE variant achieved excellent results, particularly when fine-tuned on minimal training datasets.

Conclusions:

• CPAE effectively extracts both global and local information from EHR data, outperforming existing models.
• The AtCPAE variant shows significant promise for data-scarce scenarios in health status prediction.
• Future research directions include multi-task learning integration and expanding the model to incorporate a larger number of EHR variables.