Practical considerations in the analysis of complex sample survey data

Large scale sample surveys often provide fertile ground for analyses by epidemiologists. Recently, survey organizations such as the National Center for Health Statistics and the United States Bureau of the Census have distributed data from large surveys to interested investigators via CD-ROM. Confronted by the richness of such databases and the historic relative lack of availability of suitable software to appropriately account for the survey design, researchers have often simply ignored the complexities of the survey and analyzed the data as if they resulted from a simple random sample. The availability of modern programs such as STATA and SUDAAN provides data analysts with the new analytical capabilities to perform design-based analyses whenever appropriate. We used data from the NHANES III and the PAQUID study to illustrate the ease of performing design-based analyses. We also compared results of analyses under both model-based and design-based scenarios. When data from complex sample surveys were analyzed using both model-based and design-based strategies, differences in point estimates and standard errors of means, regression coefficients and odds ratios were found. The differences in regression coefficients and odds ratios between the two strategies were not as great as the differences in means. The potential for differences and the availability of survey analysis software should encourage researchers to use design-based techniques to analyze data from complex sample surveys more appropriately.