Can small-area analysis detect variation in surgery rates? The power of small-area variation analysis

A variety of statistical methods can be used in small-area analysis to test whether there is more variation than would be expected by chance alone. However, the power of these methods to detect existing variation has never been studied. The authors used data regarding back surgery in Washington State to suggest several types of variation that might exist (alternative hypotheses), and then used computer simulation to determine the power, or the probability of detecting this variation. The chi-square test had the highest power of all methods considered against most alternative hypotheses. Power is higher if there are no multiple admissions, rates are higher, and counties have larger or similar population size. Problems of accounting for multiple admissions, adjustment for age and sex, choosing the optimum size of small areas, and detection of outliers also are discussed.