Missing data imputation in clinical trials using recurrent neural network facilitated by clustering and oversampling

Area of Science:

• Biostatistics
• Machine Learning in Clinical Research

Background:

• Complex missing data patterns (MAR and MNAR) challenge traditional statistical models in clinical practice.
• Existing methods often rely on strict assumptions, potentially leading to unreliable estimates in realistic scenarios.

Purpose of the Study:

• To develop a machine learning-based framework for accurate missing data prediction in complex scenarios.
• To handle missing not at random (MNAR) data using imbalanced learning techniques.
• To improve the reliability and accuracy of missing data handling in clinical trial analysis.

Main Methods:

• Utilized recurrent neural networks (RNN) for longitudinal data modeling.
• Employed k-means clustering on trajectories for oversampling MNAR data.
• Applied bootstrap aggregating to enhance prediction accuracy and account for uncertainty.

Main Results:

• The proposed RNN-based method demonstrated powerful predictive capabilities for longitudinal data.
• Accurate individual and population-level predictions were achieved for both MAR and MNAR data.
• Reduced bias in treatment effect estimation compared to standard and classic models.

Conclusions:

• The machine learning framework offers accurate individual predictions for complex missing data.
• The method effectively reduces bias in treatment effect estimation, outperforming traditional approaches.
• Encourages the integration of machine learning for missing data analysis in randomized clinical trials.