Comparing two machine learning approaches in predicting lupus hospitalization using longitudinal data

Systemic lupus erythematosus (SLE) is a heterogeneous autoimmune disease characterized by flares ranging from mild to life-threatening. Severe flares and complications can require hospitalizations, which account for most of the direct costs of SLE care. This study investigates two machine learning approaches in predicting SLE hospitalizations using longitudinal data from 925 patients enrolled in a multicenter electronic health record (EHR)-based lupus cohort. Our first Differential approach accounts for the time dependencies in sequential data by introducing additional lagged variables between consecutive time steps. We next evaluate the performance of LSTM, a state-of-the-art deep learning model designed for time series. Our experimental results demonstrate that both methods can effectively predict lupus hospitalizations, but each has its strengths and limitations. Specifically, the Differential approach can be integrated into any non-temporal machine learning algorithms and is preferred for tasks with short observation periods. On the contrary, the LSTM model is desirable for studies utilizing long observation intervals attributing to its capability in capturing long-term dependencies embedded in the longitudinal data. Furthermore, the Differential approach has more options in handling class imbalance in the underlying data and delivers stable performance across different prognostic horizons. LSTM, on the other hand, demands more class-balanced training data and outperforms the Differential approach when there are sufficient positive samples facilitating model training. Capitalizing on our experimental results, we further study the optimal length of patient monitoring periods for different prediction horizons.