Determining relative importance of variables in developing and validating predictive models.

Area of Science:

• Biostatistics
• Medical Informatics
• Predictive Modeling

Background:

• Automated model selection in multiple regression is common but often neglects predictor importance and model validation.
• Assessing model stability, predictive accuracy, and generalizability is crucial for reliable scientific findings.

Purpose of the Study:

• To propose and compare bootstrapping and random split strategies for ordering predictors by importance.
• To develop a method for building stable and generalizable predictive models based on predictor importance.
• To validate predictive models using proposed selection and validation strategies.

Main Methods:

• Employed bootstrapping and random data splitting to assess predictor importance and model stability.
• Utilized relative increase in explained variation and ROC curve analysis for model development.
• Applied methods to a case study predicting low-risk acute lymphoblastic leukemia (ALL) and Tumor Lysis Syndrome (TLS), and a prostate cancer dataset.

Main Results:

• Age identified as the most stable predictor of TLS (selected 100% via bootstrapping, 99% via random split).
• White blood cell count (WBC) is the second most important predictor for TLS.
• Developed a low-risk TLS prediction group: children <10 years, no T-cell immunophenotype, WBC < 20x10^9/L, palpable spleen < 2 cm.
• Gleason score and digital rectal exam identified as key predictors for prostate cancer penetration.

Conclusions:

• Bootstrap re-sampling and random split techniques effectively assess predictor importance and model reproducibility.
• Proposed methods facilitate the development of stable, reproducible, and high-performing predictive models.
• The approach serves as a valuable tool for validating predictive models, supported by existing clinical studies.