Agreement in classifying bloodstream infections among multiple reviewers conducting surveillance

Mandatory reporting of healthcare-associated infections (HAIs) is increasing. Evidence for agreement among different reviewers applying HAI surveillance criteria is limited. We aim to characterize agreement among infection preventionists (IPs) conducting surveillance for central line-associated bloodstream infection (CLABSI) with each other and as compared with simplified laboratory-based definitions.

Abstracted electronic health records were assembled from inpatients with positive blood cultures at a tertiary-care Veterans Affairs (VA) hospital over a 5-year period. Identical patient records were made available to VA IPs from different facilities to report on CLABSI using their usual surveillance methods. Positive blood cultures were also evaluated using laboratory-based definitions. Standard indices of interrater agreement, expressed as a κ statistic, were computed between IPs, and between IPs and simplified laboratory-based methods.

Overall, 114 patient records were reviewed by 18 IPs, the majority of whom specified they followed National Healthcare Safety Network criteria. The overall agreement among IPs by κ statistic was 0.42 (standard error [SE], 0.06). IPs had better agreement with a simple laboratory-based definition with an average κ of 0.55 (SE, 0.05). The proportion of patient records that 18 IPs reported with CLABSI ranged from 14% to 39% (overall mean, 28% with a coefficient of variation of 25%). When simple laboratory-based methods were applied to different sets of patient records, classification was more consistent with CLABSI assigned in a proportion ranging from 36% to 42% (overall mean, 39%).

Reliability of IP-conducted surveillance to identify HAI may not be ideal for public reporting goals of interhospital comparisons.