Clinical Implementation of an AI Algorithm for Substance Misuse Screening in Hospitalized Adults

Manual inpatient screening for substance misuse is labor-intensive and inconsistently applied. Evaluation of artificial intelligence (AI)-assisted screening during clinical implementation is needed to determine clinical and economic performance.

To assess whether an AI-based screening program with the Substance Misuse Algorithm for Referral to Treatment Using Artificial Intelligence (SMART-AI) maintained delivery of addiction-related services compared with manual screening and to evaluate readmissions and costs.

Prospective, quasi-experimental pre-post study at a large academic medical center in Chicago, Illinois between 2022 and 2025. The pre-implementation period (manual screening) included 31,432 hospitalizations, and the post-implementation period with AI augmentation (SMART-AI) included 33,564.

During the post-implementation period, SMART-AI screened clinical documentation within 24 hours of admission to identify patients at-risk for a substance use disorder and notified the Substance Use Intervention Team. In the pre-implementation period, screening relied on manual processes, with nurses and social workers screening with standardized questionnaires.

The primary outcome was receipt of ≥1 addiction-related service (initiation or adjustment of medication for alcohol or opioid use disorder; brief intervention/motivational interviewing; naloxone dispensing; or a completed addiction medicine consultation). The prespecified noninferiority margin was -0.5 percentage points (1-sided α = 0.025). Secondary outcomes included 6-month readmission, discharge against medical advice, and program costs.

Addiction-related services were received in 1,189 of 31,432 hospitalizations (3.8%) during manual screening and 1,144 of 33,564 (3.4%) during SMART-AI (difference, -0.4 percentage points; 95% CI, -0.7 to -0.1; P = 0.20). The lower limit of the confidence interval was below the noninferiority margin, so noninferiority was not achieved. Six-month readmissions across all hospitalizations occurred in 9,586 patients (30.5%) in the manual period and 10,244 patients (30.5%) in the SMART-AI period (P = 0.95), and discharge against medical advice did not differ (1.3% v. 1.1%). Among patients who received a SUIT intervention (n = 2,296), 6-month readmission occurred in 41.3% (485/1,175) during usual care versus 37.0% (415/1,121) during SMART-AI (odds ratio: 0.86, 95% CI: 0.73-1.03, p=0.10). Program costs over a 1-year period were $6,166.71 lower after SMART-AI automation.

AI-assisted screening did not meet the prespecified non-inferiority criterion for maintaining service delivery, but it was associated with maintaining secondary outcomes among patients screened for substance use disorder with lower program costs. Findings support the feasibility and potential value of automated screening at scale.

ClinicalTrials.gov Identifier: NCT03833804.