Comparison of Random Forest and Stepwise Regression for Variable Selection Using Low Prevalence Predictors: A case...

Area of Science:

• Clinical Informatics
• Biostatistics
• Machine Learning in Healthcare

Background:

• Variable selection is crucial for identifying predictive factors in clinical data.
• Low prevalence predictors (LPPs) are common but understudied in variable selection.
• This study addresses LPPs in a paediatric sepsis screening tool.

Purpose of the Study:

• Compare Random Forest (RF) and stepwise regression (SWR) for variable selection.
• Evaluate the impact of LPPs on model performance.
• Assess predictor selection with varying prevalence thresholds.

Main Methods:

• Compared RF against forward and backward SWR for variable selection.
• Utilized a paediatric sepsis screening dataset with numerous LPPs.
• Assessed model performance via Area Under the Curve (AUC) and retained variables.
• Conducted a simulation study on predictor prevalence impact.

Main Results:

• RF retained 22 predictors (14 LPPs), SWR retained 17 (10 LPPs).
• Both models achieved similar predictive performance (RF AUC: 0.79, SWR AUC: 0.80).
• Simulation showed differing variable importance and selection with increasing prevalence thresholds for both methods.

Conclusions:

• RF selected more LPPs than SWR, but predictive performance was comparable.
• Both RF and SWR are suitable for variable selection with LPPs when applied correctly.
• Model performance is robust even with a substantial number of LPPs in small candidate predictor sets.