Reproducing real-world clinical prediction models using the DIVE platform: A comparative validation study across...

Area of Science:

• Health Informatics
• Biostatistics
• Machine Learning in Healthcare

Background:

• Real-world evidence (RWE) generation from clinical data is crucial for healthcare insights.
• Existing analytical methods can be complex and time-consuming.
• The need for reproducible and scalable RWE analytics platforms is growing.

Purpose of the Study:

• To evaluate the performance and reproducibility of the Python-based Data Insight Validation Engine (DIVE).
• To assess DIVE's capability in generating RWE by replicating three published clinical studies.
• To compare DIVE's results with conventional statistical environments.

Main Methods:

• DIVE was used to replicate studies on chronic kidney disease (CKD), chronic obstructive pulmonary disease (COPD), and severe asthma prediction.
• Machine learning (ML)-based Generalized Additive 2 Model (GA2M) and Cox-based regression models were employed.
• Data from over one million patients in Italian primary care were analyzed, including external temporal validation.

Main Results:

• DIVE demonstrated high fidelity in replicating published findings across all three studies.
• CKD model: AUC 89.2% (DIVE) vs. 89.3% (original).
• COPD model: AUC 65.5% vs. 66%; Severe Asthma model: AUC 71.9% vs. 72.5%.

Conclusions:

• DIVE is a reliable, scalable, and interoperable solution for RWE analytics.
• The platform shows equivalence with traditional analytic methods, supporting data reproducibility.
• Future development could enhance DIVE's utility through federated analyses and broader interoperability.