A scalable approach for developing clinical risk prediction applications in different hospitals

Area of Science:

• Clinical informatics
• Machine learning in healthcare
• Electronic Health Records (EHR) systems

Background:

• Machine learning (ML) is widely used for predicting acute clinical events.
• Existing prediction models are often limited to a single hospital and event.
• Extending these models to different hospitals and diseases remains a challenge.

Purpose of the Study:

• To provide a scalable solution for developing and deploying clinical risk prediction models.
• To extend ML model development to multiple diseases.
• To enable deployment across different Electronic Health Records (EHR) systems.

Main Methods:

• A generic process for clinical risk prediction model development was defined.
• An automated model generation process was created using a calibration tool.
• Risk prediction models for delirium, sepsis, and acute kidney injury (AKI) were generated at four hospitals.

Main Results:

• Delirium risk models achieved an average AUROC of 0.82 (admission) and 0.95 (discharge).
• Sepsis models achieved an average AUROC of 0.88 (admission) and 0.95 (discharge).
• AKI models achieved an average AUROC of 0.85 (admission) and 0.92 (discharge).

Conclusions:

• A scalable method for developing and deploying clinical risk prediction models was described.
• The feasibility was demonstrated by developing models for three diseases across four hospitals.
• The approach relies on syntactic interoperability between EHRs, not semantic interoperability.