A Systematic Review and Meta-analysis of Externally Validated Epic Clinical Decision Support Tools.

Area of Science:

• Health Informatics
• Clinical Decision Support Systems
• Electronic Health Records

Background:

• Clinical Decision Support (CDS) tools integrated with Electronic Health Records (EHRs) are increasingly used in clinical practice.
• Epic Systems EHR, with over 325 million patient records, offers numerous proprietary predictive models.
• There is a lack of systematic reviews evaluating the real-world performance of Epic's CDS tools against vendor claims.

Purpose of the Study:

• To systematically review and analyze external validation studies of Epic's proprietary CDS tools.
• To compare the real-world performance of Epic CDS tools with vendor-reported metrics.
• To assess the generalizability and performance variations of Epic CDS tools across different healthcare settings.

Main Methods:

• Prospective registration on PROSPERO (CRD420251148571).
• Systematic literature search of PubMed, Scopus, and Embase (January 2018 to August 2025).
• Meta-analysis of Area Under the Receiver Operating Characteristic Curve (AUROC) using random-effects models and assessment of heterogeneity (I²).

Main Results:

• Twenty-two studies involving over 2.3 million patients and 34 sites were included.
• Pooled AUROC values ranged from 0.62 (Epic Risk of Patient No-Show) to 0.79 (Epic Deterioration Index).
• Significant performance discrepancies were observed for Epic Sepsis Model, Epic Unplanned Readmission Model, and Epic End-of-Life Care Index compared to Epic's reported ranges, with high heterogeneity across all models (I² ≥ 93%).

Conclusions:

• Epic's CDS tools demonstrate modest real-world performance, with none achieving an AUROC above 0.79.
• Three specific models (ESM, EURM, EEOL-CI) underperformed compared to Epic's reported performance metrics.
• The substantial heterogeneity across sites highlights the critical need for local validation of Epic CDS tools prior to clinical implementation.