Using Encounter-Level Data for Risk-Adjustment of Antimicrobial Use Comparisons: Feasibility and Variable Selection

External comparisons of hospital antimicrobial use (AU), risk-adjusted using encounter characteristics, may better inform antimicrobial stewardship program strategy. Barriers to encounter-level modeling include feasibility of data collection and defining optimal methods for selecting input variables for risk-adjustment purposes.

We measured achievements in sharing validated, encounter-level AU data among a multisystem hospital collaborative. Then, we performed retrospective analyses to compare variable selection strategies for AU risk-adjustment models. Electronic health record data from 50 US hospitals from 2020 to 2021 were split for model training and testing. Four input variable strategies were compared: (1) diagnosis-related group categories, (2) adjudicated Elixhauser comorbidity categories, (3) agnostic strategy including all diagnosis and procedure categories from AHRQ's Clinical Classification Software Refined (CCSR), and (4) adjudicated strategy where CCSR categories not appropriate for risk-adjustment were excluded by expert consensus. Gradient-boosted machine tree-based models estimated antibacterial days of therapy (DOT). Accuracy was measured for each strategy using mean absolute error (MAE); correlation plots compared model estimates and observed DOT among testing encounters. The top 20 most influential variables were defined using model variable importance.

Fifty of 76 hospitals successfully shared validated datasets using local resources. MAE was lowest for modeling strategies with larger numbers of CCSR inputs. Agnostic and adjudicated strategies had highly correlated estimates and similar influential variables.

Expert adjudication required personnel effort and potentially introduced biases, yet did not produce results different from an agnostic approach. Risk-adjustment incorporating large encounter-level data and machine learning may prove feasible and meaningful in future hospital AU assessments.