The magnitude and variability of design effects for community intervention studies

Despite inadequate replication of treatment and comparison groups in community intervention studies, suitable estimates of variance can be obtained by multiplying the simple random sampling variance by a design effect, which can be estimated from other sources. To study the stability of these estimates, the authors performed a detailed investigation of the magnitude and sources of variability of design effects for age-adjusted mortality rates for selected causes of death separately for each of 44 US states. The authors made use of age-sex-race-county-specific data from the Compressed Mortality File generated by the National Center for Health Statistics. Design effects were calculated as the ratio of a cluster (county) sampling variance to the simple random sampling variance. The effects of sex, state, and year on the magnitude of the design effects were investigated by analysis of variance. Design effects were 2.33, 1.66, 1.24, 1.08, and 1.06 for ischemic heart disease, stroke, and cancers of the lung, breast, and colon, respectively. The largest source of variability was state; 35-70% of this variability could be attributed to the states' differing population sizes. The effect of sex was minimal. These results are relevant to the planning and analysis of community intervention studies.