Metrics Selection and Risk Adjustment Methods to Benchmark Inpatient Antibiotic Use

The Centers for Disease Control and Prevention offers a standardized antimicrobial administration ratio (SAAR) as an evaluation metric for inpatient antibiotic use through rankings and peer comparisons (ie, benchmarking). However, the SAAR model only accounts for facility- and unit-level factors without considering the hierarchical nature of the health care data, and it does not directly reflect patient-level factors or stewardship efforts to avoid overly broad-spectrum therapy.

To examine the use of antimicrobial use risk adjustment methods and choice of basic metrics (eg, days of therapy [DOT] and days of antimicrobial spectrum coverage [DASC], which do not and do consider antimicrobial spectrum, respectively) in hospital benchmarking.

This retrospective cohort study was conducted using data from 117 acute care hospitals within the Veterans Health Administration (VHA) system. All patients admitted between January 1, 2021, and December 31, 2023, were included.

Monthly antibiotic use was measured with 2 basic metrics and risk adjustment models created using baseline data for 2021 to 2022. Hospitals were benchmarked for 2023 use via 3 methods: (1) unadjusted comparison, (2) risk adjustment with hospital- and unit-level factors with single-level negative binomial regression models (method 1, similar in approach to the SAAR), and (3) risk adjustment with hospital-, unit-, and patient-level factors with hierarchical zero-inflated negative binomial regression models (method 2).

This study included 736 810 patients (median age, 70 [IQR, 61-76] years; 94.7% male). There was wide variability in unadjusted antibiotic use among hospitals (median, 477 [IQR, 420-523] DOT per 1000 days present [DP]; and median, 3115 [IQR, 2739-3602] DASC per 1000 DP). Risk adjustments with methods 1 and 2 resulted in moderate ranking changes, but there were only weak correlations between benchmarking results by the 2 methods (τB = 0.43 for DOT and 0.44 for DASC). The choice of basic metrics with or without consideration of antimicrobial spectrums (DOT vs DASC) had a modest correlation after risk adjustment (τB = 0.84).

In this cohort study of the nationwide VHA system, there were substantial differences in risk-adjusted benchmarking results between models with only hospital- and unit-level factors and models with hospital-, unit-, and patient-level factors. Future studies should evaluate whether these models with higher content validity also have better construct validity and can inform hospitals and stewardship programs about their objective performance compared with other programs.