Validation of algorithms for identifying outpatient infections in MS patients using electronic medical records

Background Our multiple sclerosis (MS) stakeholder groups expressed concerns about whether MS disease-modifying therapies (DMTs) increase the risk of specific outpatient infections. Validated methods for identifying the risk of these selected outpatient infections in the general population either do not exist, exclude the clinically important possibility of recurrent infections, or are inaccurate, largely because existing studies relied primarily on International Classification of Diseases (ICD) codes to identify infectious outcomes. Additionally, no studies have validated methods among the MS population, where some MS symptoms can be mistaken for infections (e.g., urinary tract infections (UTIs)). Objective To utilize multiple data elements in the electronic health record (EHR) to improve accurate identification of selected outpatient infections in an MS cohort and general population controls. Methods We searched Kaiser Permanente Southern California's EHR based on ICD-9/10 codes for specified outpatient infections from 1/1/2008-12/31/2018 among our MS cohort (n=6000) and 5:1 general population controls matched on age, sex, and race/ethnicity (n=30,010). Random sample chart abstractions from each group were used to identify common coding errors for outpatient pneumonia, upper and lower respiratory tract infection, UTIs, herpetic infections (herpes zoster (HZ), herpes simplex virus (HSV)), fungal infections, otitis media, cellulitis, and influenza. This information was used to define discrete infectious episodes and to identify the algorithm with the highest positive predictive value (PPV) after supplementing the ICD-coded episodes with radiology, laboratory and/or pharmacy data. Results PPVs relying on ICD codes alone were inaccurate, particularly for identifying recurrent herpetic infections (HZ (42%) and HSV (60%)), UTIs (42%) and outpatient pneumonia (20%) in MS patients. Defining and validating episodes improved the PPVs for all the selected infections. The final algorithms' PPVs were 80-100% in MS and 75-100% in the general population, after including dispensed treatments (UTI, herpetic infections and yeast vaginitis), timing of dispensed treatments (UTI, herpetic infections and yeast vaginitis), removal of prophylactic antiviral use (herpetic infections), and inclusion of selected laboratory (UTIs) and imaging results (pneumonia). The only exception was outpatient pneumonia, where PPVs improved but remained ≤70%. There were no significant differences in the PPVs for the final algorithms between the MS and general population. Conclusions Provided herein are accurate and validated algorithms that can be used to improve our understanding of how the risk of recurrent outpatient infections are influenced by MS treatments, MS-related disability, and co-morbidities. Findings from such studies will be important in helping patients and clinicians engage in shared decision-making and in developing strategies to mitigate risks of recurrent infections.