Falsification of a single species hypothesis using the coefficient of variation: a simulation approach

Dental variation remains an important criterion for assessing whether a morphologically homogeneous fossil primate sample includes more than one species. The Coefficient of Variation (CV) has commonly been used to compare variation in a fossil sample of unknown taxonomic composition with that of extant single-species samples, in order to determine whether more than one species might be present. However, statistical tests for differences between fossil and single species reference sample CVs often lack power, because fossil samples are usually small and confidence limits of the CV are consequently large. The present study presents a new methodology for using the CV to test the hypothesis that a sample represents only one species. Simulated sampling distributions of single-species and pooled-species CVs are generated based on variation observed in dental samples of extant Cercopithecus species. These simulated distributions are used to test a single-species hypothesis for 13 different combinations of two or three sympatric Cercopithecus species across four dental characteristics at different sample sizes. Two different ways to generate the reference value of the CV are used. Results show the proposed methodology has substantially greater power than previous methods for detecting multiple-species composition, while maintaining an acceptable Type I error rate. Results are also presented concerning the dependence of power on sample size and on the average difference between means in a pooled-species combination.