Quality of MBSAQIP data: bad luck, or lack of QA plan?

National clinical registries are commonly used in clinical research, quality improvement, and health policy. However, little is known about methodological challenges associated with these registry analyses that could limit their impact and compromise patient safety. This study examined the quality of Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (MSBASQIP) data to assess its usability potential and improve data collection methodologies.

We developed a single flat file (n = 168,093) using five subsets (Main, BMI, Readmission, Reoperation, and Intervention) of the 2015 MBSAQIP Participant User Data File (PUF). Logic and validity tests included (1) individual profiles of patient's body mass index (BMI) changes over time, (2) individual patient care pathways, and (3) correlation analysis between variable pairs associated with the same clinical encounters.

8888 (5.3%) patients did not have postoperative weight/BMI data; 20% of patients had different units for preoperative and postoperative weights. Postoperative weight measurements ranged between - 71 and 132% of preoperative weight. There were 325 (3.7%) hospital readmissions reported on the day of or day after MBS. The self-reporting of "emergency" vs. "planned" interventions did not correlate with the type of procedure and its indication. Up to 20% of data could potentially be unused for analysis due to data quality issues.

Our analysis revealed various data quality issues in the 2015 MBSAQIP PUF related to completeness, accuracy, and consistency. Since information on where the surgery was performed is lacking, it is not possible to conclude whether these issues represent data errors, patient outliers, or inappropriate care. Including automated data checks and biomedical informatics oversight, standardized coding for complications, additional de-identified facility and provider information, and training/mentorship opportunities in data informatics for all researchers who get access to the data have been shown to be effective in improving data quality and minimizing patient safety concerns.