Evaluating Equity in Usage and Effectiveness of the CONCERN Early Warning System

The CONCERN Early Warning System (CONCERN EWS) is an artificial intelligence-based clinical decision support system (AI-CDSS) for the prediction of clinical deterioration, leveraging signals from nursing documentation patterns. While a recent multisite randomized controlled trial (RCT) demonstrated its effectiveness in reducing inpatient mortality and length of stay, evaluating implementation outcomes is essential to ensure equitable results across patient populations.This study aims to (1) assess whether clinicians' usage of the CONCERN EWS, as measured by CONCERN Detailed Prediction Screen launches, varied by patient demographic characteristics, including sex, race, ethnicity, and primary language; (2) evaluate whether CONCERN EWS's effectiveness in reducing the risk of in-hospital mortality varied across patient demographic groups.We conducted a retrospective observational analysis of electronic health record log files and clinical outcomes from a multisite, pragmatic, cluster-RCT involving four hospitals across two health care systems. Equity in usage was assessed by comparing CONCERN Detailed Prediction Screen launches across demographic groups, and effectiveness was examined by comparing the risk of in-hospital mortality between intervention and usual care groups using Cox proportional hazards models adjusted for patient characteristics.Clinicians' CONCERN Detailed Prediction Screen launches did not significantly differ by patients' demographic characteristics, suggesting equitable usage. The CONCERN EWS was significantly associated with reduced risk of in-hospital mortality overall (adjusted hazard ratio [HR] = 0.644, 95% CI: 0.532-0.778, p < 0.0001), with consistent effectiveness across most groups. Notably, patients whose primary language was not English experienced a greater reduction of mortality risk compared to patients whose primary language was English (adjusted HR = 0.419, 95% CI: 0.287-0.610, p = 0.0082).This study presents a case of evaluating equity in AI-CDSS usage and effectiveness, contributing to the limited literature. While findings suggest equitable engagement and effectiveness, ongoing evaluations are needed to understand the observed variability and ensure responsible implementation.