Combining machine learning and conventional statistical approaches for risk factor discovery in a large cohort study.

Area of Science:

• Biomedical Informatics
• Machine Learning
• Epidemiology

Background:

• Identifying health risk factors is crucial for public health.
• Large biomedical datasets offer vast potential but pose analytical challenges.
• Existing methods may struggle with non-linearity and interactions in complex data.

Purpose of the Study:

• To develop and validate a hypothesis-free machine learning pipeline for efficient risk factor discovery.
• To identify novel and known predictors of mortality in a large population cohort.
• To account for non-linearity and variable interactions in risk factor analysis.

Main Methods:

• Utilized gradient boosting decision trees (GBDT) for initial predictor screening from 11,639 variables.
• Employed SHAP (SHapley Additive exPlanations) values for feature attribution and importance.
• Validated findings using Cox regression models with false discovery rate control for confounder adjustment.

Main Results:

• Identified 193 potential mortality risk factors using the GBDT-SHAP pipeline.
• Confirmed associations with mortality for 166 predictors after adjusting for confounders.
• Health-related factors constituted 60% of the total variable importance.

Conclusions:

• The GBDT-SHAP pipeline offers an efficient and pragmatic approach for hypothesis-free risk factor identification.
• The method successfully identified known mortality risk factors and potential new ones.
• This pipeline can uncover relevant predictors within large, complex biomedical datasets.