Towards Reliable Prediction: A Bayesian Continual Learning Approach for Clinical Time-Series Data

Deep learning models are increasingly used for making predictions based on clinical time-series data, but model generalization remains a challenge. Continual learning approaches, which preserve representations while learning new distributions, are suitable for addressing this challenge. We propose Continual Bayesian Long Short Term Memory (C-BLSTM), a continual learning algorithm based on the Bayesian LSTM model for domain incremental learning. C-BLSTM continually learns a sequence of tasks by combining architectural pruning, variational inference-based regularization, and coreset replay strategies. In extensive experiments on two public electronic medical record datasets for mortality prediction, we show that C-BLSTM outperforms many state-of-the-art continual learning approaches. Further, we apply the C-BLSTM to two real-world clinical time series datasets for prediction of readmission risk in patients with heart failure and glycated haemoglobin outcomes in patients with type 2 diabetes. First, we show that these datasets exhibit domain incremental characteristics with significant drifts in their marginal distributions and moderate drifts in their conditional distributions. Then, we demonstrate that the C-BLSTM improves generalization in five diverse real-world scenarios spanning temporal, site, device, case mix, and ethnicity shifts, both in terms of performance and reliability of predictions.