Assessing data adequacy for clinical research: reliability and validity of a surgical database

As clinical databases are utilized more frequently for clinical research, it is essential that researchers assess the quality of databased information. While researchers have begun to report strategies to measure accuracy of databased information, knowledge remains limited. The purpose of this study was to assess the reliability and validity of databased information among selected study variables contained within a computerized coronary artery surgery clinical database using the written patient medical record as an external standard. Both reliability (N = 400) and validity (N = 100) samples were randomly selected from a databased sampling frame of 548 Medicare subjects who underwent coronary artery bypass grafting surgery in 1998. Reliability assessed by consistency rates were age (95%), race (94%), gender (99%), congestive heart failure (CHF) (60.5%), angina (91.5%), renal insufficiency (82%), hypertension (91.7%), diabetes mellitus (93.7%), chronic obstructive pulmonary disease (COPD) (75.5%), clinical status (97%), American Society of Anesthesiologists classification (99%), prior peripheral vascular surgery (15.5%), prior CABGS (99%), and duration of mechanical ventilation (87.5%). These percentages reflected a large portion of missing data for CHF, COPD, and prior peripheral vascular surgery. Validity assessed by sensitivity and specificity analyses were all greater than 80%. The majority of computerized databased information among selected study variables was the same information recorded in the written patient medical record. Using the same external standard to assess both reliability and validity was a significant limitation of this study, which resulted in the same measure of data adequacy by utilizing differing statistical methods.