Development and Validation of an Electronic Health Record-Based Algorithm for Identifying Patients With Long-Term...

Health care providers must carefully monitor patients receiving long-term opioid therapy (LTOT) to minimize risks and maximize benefits. Yet, algorithms to support intervention during patient encounters are lacking, with accurate LTOT identification in routine care being the essential first step.

This study aims to develop and validate an LTOT identification algorithm using electronic health record (EHR) data.

In this cross-sectional study, we used 2016-2021 OneFlorida+ EHR data linked with Florida Medicaid claims to identify patients aged ≥18 years who received opioid prescriptions. The main outcome was the first LTOT episode in the algorithm development (2016-2018) and validation (2019-2021) periods. A Medicaid claims-based LTOT algorithm served as the reference standard, defined as ≥90 days of continuous opioid use with ≤15-day gaps. Given strong correlations among covariates, an elastic net regression model was applied to identify LTOT episodes in EHR data using patient characteristics, clinically relevant features, and medication use, and to evaluate the model's classification performance. We randomly split the 2016-2018 cohort into development and internal validation datasets (2:1 ratio), stratified by LTOT incidence. External validation was performed using 2019-2021 data.

Among 64,206 eligible patients identified in 2016-2018 (mean age 35.7, SD 12.3 years; 51,421/64,206, 80.1% female), a total of 8899 (13.9%) had LTOT. Among 50,009 eligible patients identified in 2019-2021 (mean age 37.3, SD 12.5 years; 39,866/50,009, 79.7% female), a total of 6000 (12%) had LTOT. The model selected 29 out of 131 candidate features. Among 2967 individuals with LTOT in the 2016-2018 OneFlorida+ internal validation dataset, a total of 2176 (73.3%) individuals were identified in the top 3 deciles of risk scores. The model achieved a C-statistic of 0.83 (95% CI 0.82-0.84), with 73.4% (95% CI 71.8%-75%) sensitivity, 76.8% (95% CI 76.2%-77.4%) specificity, 33.8% (95% CI 33.1%-34.6%) precision, 76.3% (95% CI 75.8%-76.9%) accuracy, and an F₁-score of 0.46. In the 2019-2021 OneFlorida+ external validation dataset, a total of 75.5% (4527/6000) individuals were correctly captured in the top 3 risk subgroups. The model achieved a C-statistic of 0.83 (95% CI 0.83-0.84), with 78.8% (95% CI 77.8%-79.9%) sensitivity, 73.3% (95% CI 72.9%-73.7%) specificity, 28.7% (95% CI 28.3%-29.1%) precision, 73.9% (73.6%-74.3%) accuracy, and an F₁-score of 0.42.

The EHR-based LTOT algorithm showed comparable accuracy to the claims-based reference and may support risk stratification and inform decision-making during clinical encounters.