Random generalized linear model: a highly accurate and interpretable ensemble predictor.

Area of Science:

• Machine Learning
• Statistical Modeling
• Bioinformatics

Background:

• Ensemble predictors like random forests offer high accuracy but lack interpretability.
• Generalized linear models (GLMs) are interpretable but prone to overfitting and low accuracy with forward selection.
• Combining ensemble accuracy with GLM interpretability is a key research challenge.

Purpose of the Study:

• To develop and evaluate a novel predictor that integrates the strengths of ensemble methods and GLMs.
• To address the limitations of existing GLM-based ensemble predictors.
• To provide a highly accurate and interpretable modeling approach.

Main Methods:

• A novel bootstrap aggregated (bagged) GLM predictor, termed random generalized linear model (RGLM), was developed.
• RGLM incorporates randomness and instability through methods like random subspace and forward variable selection.
• Evaluations were conducted using extensive genomic datasets, UCI machine learning benchmarks, and simulations.

Main Results:

• The RGLM predictor frequently outperformed alternative methods, including random forests and penalized regression models.
• RGLM provides robust variable importance measures.
• A "thinned" ensemble predictor derived from RGLM maintained excellent predictive accuracy with fewer features.

Conclusions:

• RGLM offers a state-of-the-art prediction method combining high accuracy and interpretability.
• It provides feature importance and out-of-bag estimates similar to random forests.
• The RGLM methodology is available in the open-source R software package randomGLM.