Accuracy of data collected by surgical residents

Clinician records are the primary information source for assessing the quality of facial injury care, billing, risk management, planning of health services, and health-system management and reporting. Inaccuracies obscure outcomes assessment and affect the planning of health services. We sought to determine the accuracy of the clinician collected data by comparing them to similar information elicited by professional interviewers.

We abstracted admissions data from the medical records of 185 patients treated for orofacial injury between January 2005 and January 2007. Clinician data on sociodemographics and substance use were compared with similar information elicited by trained research staff as part of a prospective study.

The accuracy of the clinician data sets varied considerably depending on the variable. Concordance with the interviewer data sets was highest for age (paired t test P = .09), gender (kappa = 1), and ethnicity (kappa = .84) but dropped off considerably for marital status (kappa = .22) and alcohol (kappa = .18) and drug use (kappa = .16). The missing data per variable ranged from 4.5% (gender) to 46.9% (employment and education).

Although more research is needed to evaluate the cause of inaccuracies and the relative contributions of patient, provider, and system level effects, it seems that significant inaccuracies in administrative data are common. In particular, patient information collected by surgical residents under-reports substance use behaviors. Interventions aimed at identifying the sources and correcting these errors are necessary.