Quantitative Genetic Analysis of Baboon Facial Variation Challenges a Classic Papionin Trait

The Linnaean family of monkeys that includes baboons, macaques, and mandrills (among others) is notably characterized by large, extended muzzles that vary in lateral nasal dorsum profile. In paleontological studies, this variation is referred to as the anteorbital drop (AOD). Here, we first devise a method to quantify this phylogenetic character. We then employ quantitative genetic analyses using a MCMCglmm model of facial variation in a captive, pedigreed baboon colony (n = 934), including admixed individuals derived from at least two Papio species, from the Southwest National Primate Research Center to assess the quantitative variation of the AOD, its genetic integration relative to other aspects of facial variation, and its relative evolvability. Specifically, we test two hypotheses to assess the appropriateness of the AOD as a way to characterize lateral nasal dorsum profile variation for paleontological studies. We find that the first hypothesis of limited variation within the population is rejected, as this admixed population demonstrates a remarkable degree of AOD variation spanning what is observed across papionin genera. The second hypothesis proposes that variation in the AOD is highly heritable and highly evolvable. Our results indicate that the AOD is highly heritable, but simultaneously less evolvable than other aspects of facial variation. Additionally, sex has little influence on AOD variation, in contrast to other facial measurements. These results challenge the use of AOD as a phylogenetic character, as it likely results from a broad range of non-sexually dimorphic processes that contribute to facial length. We caution against using AOD as a discrete character in phylogenetic analyses and invite further investigation of its biological significance. This study aims to illustrate how quantitative genetic approaches can probe the evolutionary relevance of traditional traits used in paleontology.