Should We Question the External Validity of Database Studies? A Comparative Analysis of Demographics

To define the external validity of national and institutional databases for common sports medicine procedures.

Patient demographic data including age, sex, body mass index (BMI), and 4 racial categories were aggregated between 2007 and 2016 across 2 databases for 4 common sports medicine procedures: anterior cruciate ligament reconstruction, arthroscopic rotator cuff repair (RCR), partial meniscectomy (PMx), and both arthroscopic and open shoulder stabilization. The first database of interest was a prospectively collected institutional database. The second was the National Surgical Quality Improvement Program (NSQIP) database. Two-sample t tests were performed to examine mean differences (MDs) in age and BMI, and χ² testing was used to test differences in sex and race.

A total of 7,019 institutional and 108,881 NSQIP patients were examined. The NSQIP cohort was significantly older (MD, 1.40 years), included more female patients (42.60% female patients vs 35.67% female patients), and showed a different racial distribution compared with the institutional data (all P < .0001). The NSQIP PMx cohort (MD, 7.38 years) was significantly older and the NSQIP RCR cohort (MD, 1.97 years) was significantly younger than their institutional counterparts (all P < .0001). The NSQIP anterior cruciate ligament reconstruction cohort (MD, 2.53) showed a greater average BMI (P < .0001). The NSQIP RCR cohort (41.8% female patients vs 33.3% female patients) and PMx cohort (46.0% female patients vs 37.9% female patients) also included more female patients. Race was distributed variably between databases for each procedure code (all P < .0001).

Significant differences in age, BMI, sex, and race distributions were observed between an institutional database and the NSQIP database. This study underlines the importance of defining the generalizability of database research, particularly when significant demographic differences between databases may underlie differences in postoperative outcomes.

Level III, cross-sectional study.