A spreadsheet to construct power curves and clarify the meaning of the word equivalent in evaluating experiments with...

Persons conducting research trials often want to be able to declare that treatments, or particularly products, are equivalent (will provide indistinguishable results). However, all research trials can ever provide is the probability that the observed differences in an experiment were due to chance. Also, in trials in which variances are high and there are few replications, it is quite easy to declare no significant differences and equivalency. This paper describes a Microsoft Excel spreadsheet that can be used to easily construct experimental power curves. Such curves predict the proportion of experiments that would yield a given level of significance as the difference between the 2 means increases. The spreadsheet uses the mean and variances from an experiment with the Norm.inv and Rand functions of Excel to simulate outcomes from identical experiments. An experiment that declared GMO and normal feed ingredients to be equivalent was used to illustrate the application of power curves. The experiment had 12 replicate pens of broilers per treatment. The outcomes of 90,000 simulated experiments, each with the same overall variance, but 0 through 8 percent differences in treatment means, were graphed. When the published experiment purported to show equivalence, really it showed that a significant difference in growth (P < 0.05) would be expected to be detected 50% of the time if the means were different by 3.1%; a difference of 4.6% in treatment means could be detected 80% of the time by such an experiment. This Excel spreadsheet enables such a power analysis to be conducted. Easy modifications of the spreadsheet can illustrate the influence of changing the variance or number of replications on the expected power of future experiments. The economic impact of small changes in performance is also discussed.