Evaluating the generalizability of commercial healthcare claims data

Commercial healthcare claims datasets area nonrandom sample of the US population, affecting generalizability. Rigorous comparisons of claims-derived results to ground-truth data that quantify external validity bias are lacking. Our goal is to (1) quantify external validity of commercial healthcare claims data and (2) to evaluate how socioeconomic/demographic factors are related to the bias. We analyzed inpatient discharge records occurring between January 1, 2019 and December 31, 2019 in five states: California, Iowa, Maryland, Massachusetts, and New Jersey, and compared rates (per person-year) of the 250 most common inpatient procedures between claims and reference data for each target population. We used Merative MarketScan Commercial Database for the claims data and State Inpatient Databases and the US Census as reference. For a target population of all Americans, commercial healthcare claims underestimate the rate of overall inpatient discharges by 23.1%. The extent of bias varied across procedures, with the rates of ~25% of procedures being underestimated by a factor of 2. Socioeconomic factors were significantly associated with the magnitude of bias (${R}^2=69.4\%,$P < .001). When the target population was restricted to commercially insured Americans, the bias decreased substantially (1.4% of procedures were biased by more than factor of 2), but some variation across procedures remained.