Establishment and evaluation of a multicenter collaborative prediction model construction framework supporting model...

Area of Science:

• Clinical informatics
• Machine learning in healthcare
• Multicenter research networks

Background:

• Clinical prediction models are increasingly vital for healthcare decision-making.
• Existing models often lack generalizability and continuous improvement mechanisms.
• Electronic health record (EHR) data offers a rich source for data-driven prediction models.

Purpose of the Study:

• To propose a multicenter collaborative framework for constructing prediction models.
• To enhance model generalizability and enable continuous improvement.
• To ensure patient data security and privacy during model development.

Main Methods:

• Utilized a multicenter collaborative network (e.g., OHDSI) and multi-source transfer learning.
• Trained base classifiers in source hospitals and integrated them in target hospitals.
• Employed a passive-aggressive online learning algorithm for continuous model enhancement.
• Developed a prototype for colorectal cancer prognosis prediction using 70,906 patients from US and Chinese datasets.

Main Results:

• The proposed model demonstrated superior calibration (ECI = 9.294) and comparable discrimination (AUC = 0.783) compared to reference models.
• Online learning improved model performance, increasing AUC from 0.709 to 0.715 and decreasing ECI from 13.013 to 9.634.
• The framework facilitates continuous performance maintenance in clinical applications.

Conclusions:

• A novel multicenter collaborative framework for prediction model construction was proposed and validated.
• The framework enables the development of generalizable models with continuous improvement capabilities.
• Crucially, this approach avoids the need for aggregating sensitive patient-level data across institutions.