Methodological Research Informing Weighting, Combination, and Quality Assessment for the National Center for Health...

The National Center for Health Statistics's Rapid Surveys System (RSS) was established in 2022. It uses two probability-based commercial online panels, NORC at the University of Chicago's AmeriSpeak panel and Ipsos's KnowledgePanel, to enable fast data collection on emerging or priority health topics. RSS Rounds 1-4 (RSS 1-4) were collected between August 2023 and May 2024. This report describes the methodology implemented in the early rounds of RSS for weighting and variance estimation, methods for combining multiple panels, and quality assessment.

RSS 1-4 were weighted to a set of control totals for selected demographic and health variables from the National Health Interview Survey (NHIS). Estimates from the two online panels were combined using a convex combination based on effective sample sizes to produce one set of health estimates for release each round. As part of an ongoing quality evaluation, quality metrics were assessed and reported for each round and were used to inform the RSS methodology in successive rounds.

RSS 1-4 had sample sizes ranging from 7,046 (Round 2) to 8,705 (Round 4), with cumulative response rates ranging from 3.8% to 9.9% across the two panel providers. Quality metrics including nonresponse rates and breakoff rates were evaluated and found to be consistent for the two panel providers. In addition, a set of benchmark variables collected on RSS and NHIS were evaluated to assess bias. While the magnitude of bias varied by health variable and across health domains, most health variables had low bias across the four rounds.

The methodology for RSS has evolved and allowed for changes to improve the quality of the program and the resulting estimates. This report describes the early methodology implemented in the first four rounds of RSS.