Interobserver and intraobserver reliability in the collection of emergency medical services data

The collection of data by abstraction from patient records is a widely used method of research, evaluation, and registry. Since valid conclusions depend on the accuracy of the abstracted data, it is essential to examine the abstracting procedures. In this paper, we report on a study of patient data abstracted from emergency department records by nurses trained by project personnel. Twenty-five charts were selected at each of five hospitals. To test interobserver reliability, the nurses were asked to abstract all of the charts at each hospital; to test intraobserver reliability, four of the nurses each reabstracted a set of charts. The results show that even with highly trained, well motivated abstractors, there are considerable differences in the accuracy with which the variables are abstracted. Disposition from the hospital, quantitative vital signs, and blood gas values tend to be abstracted with higher reliability; whereas variables requiring judgment, such as character of vital signs or history of disease, tend to have low reliability. To improve the quality of abstracted data, we propose improved retrieval methods for hospital records, monitoring of data collection procedures, cooperation of all medical personnel providing the raw data, and careful selection of variables.