Post-Analysis of Predictive Modeling with an Epidemiological Example

Area of Science:

• Epidemiology
• Biostatistics
• Machine Learning in Healthcare

Background:

• Post-analysis of predictive models is crucial for practical application and understanding model logic, especially in high-dimensional epidemiological data.
• Investigating variations in model performance across different subject conditions is essential for robust epidemiological predictions.

Purpose of the Study:

• To present a model-independent approach for post-analysis to identify subject conditions affecting predictive model performance.
• To reveal factors contributing to 'easy' and 'difficult' cases in cardiovascular death prediction models.

Main Methods:

• Utilized Lasso logistic regression (LLR) to predict cardiovascular death using data from the Kuopio Ischemic Heart Disease Risk Factor Study (KIHD).
• Employed a multi-objective evolutionary algorithm (MOGA) to generate rules identifying subject conditions associated with model performance.
• Performed 50 independent runs of five-fold cross-validation on 2682 subjects with 950 preselected predictors.

Main Results:

• The LLR model achieved an average accuracy of 72.53% for predicting cardiovascular death.
• Post-analysis identified distinct subject categories: 'Easy' cases (95.84% accuracy), 'difficult' cases (48.11% accuracy), and remaining cases (71.00% accuracy).
• Rule analysis indicated that medication use was a significant factor complicating model performance.

Conclusions:

• The proposed model-independent approach effectively identifies subject conditions that influence predictive model performance.
• Medication emerged as a key confounding factor affecting the accuracy of cardiovascular death prediction models.
• This method provides valuable insights for improving the reliability and interpretability of epidemiological predictive models.