Addressing Measurement Error in Random Forests Using Quantitative Bias Analysis.

Area of Science:

• Machine Learning
• Biostatistics
• Data Science

Background:

• Variables in datasets are frequently measured with inaccuracies.
• The effect of such measurement errors on machine learning (ML) model predictions is often unexamined.

Purpose of the Study:

• To evaluate how measurement error in predictors impacts random-forest model performance and variable importance.
• To demonstrate the utility of quantitative bias analysis in correcting for measurement error in ML.

Main Methods:

• Assessed the impact of categorical variable misclassification on random-forest model accuracy and sensitivity using National Comorbidity Survey Replication data.
• Developed simulated datasets to verify the accuracy of quantitative bias analysis in recovering true performance and importance measures from erroneous data.

Main Results:

• Measurement error in predictors significantly distorts random-forest model performance metrics (e.g., accuracy, sensitivity).
• Variable importance measures derived from data with measurement error are also distorted.
• Quantitative bias analysis effectively recovers accurate model performance and variable importance from misclassified data.

Conclusions:

• Measurement error is a critical factor that can compromise the integrity of machine learning models.
• Quantitative bias analysis is a valuable tool for mitigating the effects of measurement error in ML studies.
• Applying bias analysis enhances the reliability of findings from machine learning models trained on real-world data.