Constructing a binary prediction model with incomplete data: Variable selection to balance fairness and precision.

Area of Science:

• Psychology
• Statistics
• Data Science

Background:

• The tension between explanation and prediction in psychology necessitates robust variable selection.
• Missing data complicate standard variable selection and penalized regression methods.
• Evaluating model performance requires considering both prediction accuracy and fairness, especially for societal implications.

Purpose of the Study:

• To explore and compare two methods for variable selection with incomplete data: bootstrap imputation-stability selection (BI-SS) and stacked elastic net (SENET).
• To evaluate the performance of BI-SS and SENET using prediction accuracy and fairness metrics across various simulation complexities.
• To assess the suitability of these methods for generalized linear models and nested data designs.

Main Methods:

• Employed bootstrap imputation-stability selection (BI-SS) on multiply imputed datasets, aggregating results via stability selection.
• Utilized stacked elastic net (SENET) by stacking imputed datasets for a single pooled model fit.
• Conducted three simulation studies with increasing complexity, evaluating performance with metrics like AUC, F1 score, and fairness criteria.

Main Results:

• BI-SS and SENET showed comparable performance for generalized linear models.
• BI-SS demonstrated superior performance in nested data designs due to computational demands of SENET with mixed-effects models.
• Both methods were successfully demonstrated on electronic health data.

Conclusions:

• BI-SS is a robust method for variable selection with incomplete data, particularly advantageous in nested data structures.
• The choice between BI-SS and SENET may depend on the data structure and model complexity.
• Further application in real-world datasets like electronic health records is validated.