Multiple imputation with sequential penalized regression.

Area of Science:

• Statistics
• Data Science
• Biostatistics

Background:

• Missing data is a pervasive challenge in research, impacting estimation and inference.
• Existing multiple imputation software struggles with datasets featuring numerous covariates with missing values.
• Maximum Likelihood Estimation (MLE) can be suboptimal with many predictors relative to sample size.

Purpose of the Study:

• To introduce mispr, a novel multiple imputation algorithm designed for complex datasets.
• To address limitations of current software in handling high-dimensional missing data.
• To improve imputation accuracy and regression analysis performance.

Main Methods:

• Developed mispr using sequential penalized regression models with ridge penalty.
• Each variable's imputation model allows for different distributional forms.
• Employed a quadratic penalty for unique parameter estimates and improved predictions, especially when predictors exceed sample size.

Main Results:

• mispr demonstrated superior performance compared to mice, VIM, and Amelia in simulation studies.
• Achieved lower mean squared imputation error (MSE), mean absolute imputation error (MAIE), and regression MSE.
• Performance gains were particularly pronounced with an increasing number of covariates, especially in small sample scenarios.

Conclusions:

• mispr offers a robust and accurate solution for multiple imputation, particularly in high-dimensional settings.
• The algorithm provides a favorable bias-variance trade-off, enhancing predictive accuracy.
• mispr is a competitive alternative to existing imputation methods, showing significant advantages in challenging data conditions.