Performance of Firth-and logF-type penalized methods in risk prediction for small or sparse binary data

Area of Science:

• Statistics
• Biostatistics
• Machine Learning

Background:

• Maximum likelihood estimation (MLE) in logistic regression struggles with small/sparse data, leading to biased estimates and convergence failures due to separation.
• Separation issues, common even with large sample sizes and strong predictors, result in overfitted models with poor predictive performance.
• Firth- and logF-type penalized regression methods are alternatives to MLE for separation problems, but their use in risk prediction is limited.

Purpose of the Study:

• To evaluate Firth- and logF-type penalized regression methods for risk prediction in small or sparse datasets.
• To compare the performance of penalized methods against standard MLE and ridge regression.
• To assess calibration, discrimination, and overall predictive performance.

Main Methods:

• An extensive simulation study was conducted to evaluate predictive performance.
• Calibration, discrimination, and overall predictive performance were assessed for each method.
• A real-world dataset with low outcome prevalence was used for illustration.

Main Results:

• MLE demonstrated poor risk prediction performance in small or sparse datasets.
• All penalized methods showed improvements in calibration, discrimination, and overall performance compared to MLE.
• LogF(1,1) penalization outperformed other methods, including Firth-type and logF(2,2), with minimal bias and good predictive accuracy. Ridge regression showed good discrimination but often resulted in underfitted models and high convergence failure rates.

Conclusions:

• The logF-type penalized method, specifically logF(1,1), is recommended for practical use in developing risk models with small or sparse datasets.
• Penalized regression offers a viable solution to the challenges posed by separation in logistic regression for risk modeling.
• LogF(1,1) provides a robust approach for accurate risk prediction when data is limited or sparse.