Benchmarking Missing Data Imputation Methods for Time Series Using Real-World Test Cases

Area of Science:

• Healthcare data science
• Biostatistics
• Machine learning in medicine

Background:

• Missing data is a significant challenge in healthcare analytics.
• Current imputation methods are often evaluated on unrealistic missing data patterns.
• Real-world missingness mechanisms (MCAR, MAR, NMAR) require robust imputation strategies.

Purpose of the Study:

• To assess the real-world accuracy of 12 imputation methods across three missing data mechanisms (MCAR, MAR, NMAR).
• To compare imputation performance on continuous glucose monitoring and heart rate time series data.
• To evaluate the impact of missingness percentages (5-30%) on imputation accuracy.

Main Methods:

• Simulated missingness in Loop (CGM) and All of Us (heart rate) datasets according to MCAR, MAR, and NMAR mechanisms.
• Tested 12 state-of-the-art and commonly used imputation methods.
• Evaluated accuracy using root mean square error (RMSE) and bias metrics across demographic groups.

Main Results:

• Imputation accuracy was significantly higher for missing completely at random (MCAR) data compared to missing at random (MAR) and not missing at random (NMAR) data.
• Linear interpolation demonstrated the lowest RMSE and minimal bias across all tested mechanisms and demographic groups.
• Existing evaluation practices may overestimate imputation method performance in real-world scenarios.

Conclusions:

• Current imputation method evaluations do not reflect real-world performance with realistic missing data patterns.
• Linear interpolation offers a reliable baseline for imputation, even with complex missingness.
• Further research should focus on developing improved evaluation methodologies and imputation techniques tailored to real-world missing data mechanisms.