A simulation test of Smith's "Degrees of freedom" correction for comparative studies

Computer simulation was used to test Smith's (1994) correction for phylogenetic nonindependence in comparative studies. Smith's method find effective N, which is computed using nested analysis of variance, and uses this value in place of observed N as the baseline degrees of freedom (df) for calculating statistical significance levels. If Smith's formula finds the correct df, distributions of computer-generated statistics from simulations with observed N nonindependent species should match theoretical distributions (from statistical tables) with the df based on effective N. The computer program developed to test Smith's method simulates character evolution down user-specified phylogenies. Parameters were systematically varied to discover their effects on Smith's method. In simulations in which the phylogeny and taxonomy were identical (tests of narrow-sense validity), Smith's method always gave conservative statistical results when the taxonomy had fewer than five levels. This conservative departure gave way to a liberal deviation in type I error rates in simulations using more than five taxonomic levels, except when species values were nearly independent. Reducing the number of taxonomic levels used in the analysis, and thereby eliminating available information regarding evolutionary relationships, also increased type I error rates (broad-sense validity), indicating that this may be inappropriate under conditions shown to have high type I error rates. However, the use of taxonomic categories over more accurate phylogenies did not create a liberal bias in all cases in the analysis performed here. The effect of correlated trait evolution was ambiguous but, relative to other parameters, negligible.