Variation in the nasal cavity of baboon hybrids with implications for late Pleistocene hominins

Hybridization is increasingly proving to be an important force shaping human evolution. Comparisons of both ancient and modern genomes have provided support for a complex evolutionary scenario over the past million years, with evidence for multiple incidents of gene exchange. However, to date, genetic evidence is still limited in its ability to pinpoint the precise time and place of ancient admixture. For that we must rely on evidence of admixture from the skeleton. The research presented here builds on previous work on the crania of baboon hybrids, focusing on the nasal cavity of olive baboons, yellow baboons, and first generation (F1) hybrids. The nasal cavity is a particularly important anatomical region for study, given the clear differentiation of this feature in Neanderthals relative to their contemporaries, and therefore it is a feature that will likely differ in a distinctive manner in hybrids of these taxa. Metric data consist of 45 linear, area, and volume measurements taken from CT scans of known-pedigree baboon crania. Results indicate that there is clear evidence for differences among the nasal cavities of the parental taxa and their F1 hybrids, including a greater degree of sexual dimorphism in the hybrids. There is also some evidence for transgressive phenotypes in individual F1 animals. The greatest amount of shape variation occurs in the anterior bony cavity, the choana, and the mid-nasopharynx. Extrapolating our results to the fossil record, we would expect F1 hybrid fossils to have larger nasal cavities, on average, than either parental taxon, with overall nasal cavity shape showing the most profound changes in regions that are distinct between the parental taxa (e.g., anterior nasal cavity). We also expect size and shape differences to be more pronounced in male F1 hybrids than in females. Because of pronounced anterior nasal cavity differences between Neanderthals and their contemporaries, we suggest that this model might be effective for examining the fossil record of late Pleistocene contact.