Training, quality assurance, and assessment of medical record abstraction in a multisite study

Clinical studies using medical record review should include careful training and quality assurance methods to enhance the reliability and validity of data obtained from the records. Because of time and budget constraints, comprehensive assessments of data quality and reliability, including masking of medical record abstractors, are not always possible. This paper describes the abstractor training and quality control methods and results of a masked medical record review study. The medical record review study was carried out within a larger multisite study of the effectiveness of screening mammography in preventing breast cancer mortality with an observation period within 1983 and 1993, with mortality follow-up through 1998. An eight-step program was developed to train medical record abstractors and monitor the quality of their work. A key follow-up component to the training protocol was a 5% reabstraction of medical records (n = 160), masked and reviewed by a second abstractor. High agreement was found between initial (unmasked) abstractors and masked abstractors for all key exposure variables (kappa ranged from 0.76 to 0.91), with no evidence of biased directionality by unmasked reviewers. Rigorous ongoing training programs for medical record abstractors provide assurance of good quality control in large multisite studies. Additionally, a masking study with a subsample of subjects may be a feasible and cost-effective alternative to the time- and cost-intensive methodological approach of masking all medical records.