Scalable Bayesian Nonparametric Method for Clinical Risk Prediction Using Large-Scale Data From Heterogeneous...

Area of Science:

• Computational statistics
• Biostatistics
• Machine learning in healthcare

Background:

• Large clinical datasets offer potential for enhanced risk prediction but pose challenges due to patient heterogeneity and data dynamics.
• Existing risk modeling techniques struggle with complex, overlapping data distributions common in healthcare.
• Bayesian nonparametric methods like Dirichlet Process Mixture Models (DPMM) are suitable for such data but are computationally intensive for large datasets.

Purpose of the Study:

• To develop a scalable framework for constructing Dirichlet Process Mixture Models (DPMMs) from large clinical datasets.
• To address the computational limitations of DPMMs for big data applications in risk prediction.
• To improve the accuracy and efficiency of risk prediction models by accounting for patient heterogeneity.

Main Methods:

• A scalable framework was developed by dividing large datasets into smaller subsets for parallel DPMM learning.
• A recentered pseudo-barycenter was used to approximate the full dataset's posterior density.
• A novel algorithm was designed for consistent clustering from subset posteriors with varying component numbers.

Main Results:

• The proposed framework demonstrated improved accuracy in predicting heart failure patient survival post-left ventricular assist device implantation compared to Cox proportional hazards and random survival forests.
• The method effectively clustered patients into distinct risk subgroups, accounting for overlapping posterior mixtures.
• Validation through simulation and a clinical case study confirmed the framework's efficacy.

Conclusions:

• The developed scalable DPMM framework offers an effective approach for analyzing large, heterogeneous clinical datasets.
• This method enhances risk prediction accuracy by adaptively clustering patients and modeling overlapping risk profiles.
• The framework provides a computationally feasible solution for applying advanced Bayesian methods to real-world clinical data challenges.