Complex sampling: implications for data analysis

Investigators in dental public health often use strategies other than simple random sampling to identify potential subjects; however, their statistical analyses do not always take into account the complex sampling mechanism. Often it is not clear whether a given strategy requires adjustment for stratification and/or cluster sampling of observations. We propose that the need for such adjustment depends on the primary study objective. As a general rule, we recommend that if the study goal is to estimate the magnitude of either a population value of interest (e.g., prevalence), or an established exposure-outcome association, adjustment of variances to reflect complex sampling is essential because obtaining appropriate variance estimates is a priority. However, if the study goal is to establish the presence of an association, especially in a preliminary investigation of novel conditions or understudied populations, obtaining appropriate variance estimates may not be of primary importance; hence, adjustment of variances for complex sampling is not always required, but often is recommended. This paper describes several types of complex sampling designs, methods of adjusting for complex sampling strategies, examples illustrating the effect of adjustment, and alternative approaches for analysis of complex samples.