The Reliability of Electronic Health Record Data Used for Obstetrical Research

Hospital electronic health record (EHR) data are increasingly being called upon for research purposes, yet only recently has it been tested to examine its reliability. Studies that have examined reliability of EHR data for research purposes have varied widely in methods used and field of inquiry, with little reporting of the reliability of perinatal and obstetric variables in the current literature.

To assess the reliability of data extracted from a commercially available inpatient EHR as compared with manually abstracted data for common attributes used in obstetrical research.

Data extracted through automated EHR reports for 3,250 women who delivered a live infant at a large hospital in the Pacific Northwest were compared with manual chart abstraction for the following perinatal measures: delivery method, labor induction, labor augmentation, cervical ripening, vertex presentation, and postpartum hemorrhage.

Almost perfect agreement was observed for all four modes of delivery (vacuum assisted: kappa = 0.92; 95% confidence interval [CI] = 0.88-0.95, forceps assisted: kappa = 0.90; 95%CI = 0.76-1.00, cesarean delivery: kappa = 0.91; 95%CI = 0.90-0.93, and spontaneous vaginal delivery: kappa = 0.91; 95%CI = 0.90-0.93). Cervical ripening demonstrated substantial agreement (kappa = 0.77; 95%CI = 0.73-0.80); labor induction (kappa = 0.65; 95%CI = 0.62-0.68) and augmentation (kappa = 0.54; 95%CI = 0.49-0.58) demonstrated moderate agreement between the two data sources. Vertex presentation (kappa = 0.35; 95%CI = 0.31-0.40) and post-partum hemorrhage (kappa = 0.21; 95%CI = 0.13-0.28) demonstrated fair agreement.

Our study demonstrates variability in the reliability of obstetrical data collected and reported through the EHR. While delivery method was satisfactorily reliable in our sample, other examined perinatal measures were less so when compared with manual chart abstraction. The use of multiple modalities for assessing reliability presents a more consistent and rigorous approach for assessing reliability of data from EHR systems and underscores the importance of requiring validation of automated EHR data for research purposes.