Bootstrap simulations to estimate relationships between Type I error, power, effect size, and appropriate sample...

The Extended Bootstrap (EB) assessment approach was developed for the examination of relationships of Type I error, power, sample size (n), and effect size (ES) for statistical tests of ecological data. The EB approach was applied to univariate and multivariate statistical analyses of a large data set collected from an ongoing, multiple stressor bioassessment study of watersheds in the Central Valley, San Francisco, and Central Coast areas of California. Benthic metrics were created that either increased or decreased monotonically with stress (toxicants or metrics indicative of habitat quality). Type I errors were stable for all statistical tests that were evaluated. The relationships between n and ES displayed patterns of "diminishing returns" for all statistical tests: i.e. an increasingly larger n was required to detect decreasingly smaller ES. Nonetheless, the n's collected across the watersheds and within a selected watershed were sufficient to detect even small correlations between representative benthic metrics and potential stressors with high power. The power and robustness of a novel method using EB and previously described statistical techniques designed to address multicollinearity were shown to approach those of simpler univariate regressions. Potential applications of the EB approach for experimental design, data assessment and interpretation, and hypothesis testing are discussed.