An assessment of statistical methods for nonindependent data in ecological meta-analyses

In ecological meta-analyses, nonindependence among observed effect sizes from the same source paper is common. If not accounted for, nonindependence can seriously undermine inferences. We compared the performance of four meta-analysis methods that attempt to address such nonindependence and the standard random-effect model that ignores nonindependence. We simulated data with various types of within-paper nonindependence, and assessed the standard deviation of the estimated mean effect size and Type I error rate of each method. Although all four methods performed substantially better than the standard random-effects model that assumes independence, there were differences in performance among the methods. A two-step method that first summarizes the multiple observed effect sizes per paper using a weighted mean and then analyzes the reduced data in a standard random-effects model, and a robust variance estimation method performed consistently well. A hierarchical model with both random paper and study effects gave precise estimates but had a higher Type I error rates, possibly reflecting limitations of currently available meta-analysis software. Overall, we advocate the use of the two-step method with a weighted paper mean and the robust variance estimation method as reliable ways to handle within-paper nonindependence in ecological meta-analyses.