Strong reproductive isolation between humans and Neanderthals inferred from observed patterns of introgression

Recent studies have revealed that 2-3% of the genome of non-Africans might come from Neanderthals, suggesting a more complex scenario of modern human evolution than previously anticipated. In this paper, we use a model of admixture during a spatial expansion to study the hybridization of Neanderthals with modern humans during their spread out of Africa. We find that observed low levels of Neanderthal ancestry in Eurasians are compatible with a very low rate of interbreeding (<2%), potentially attributable to a very strong avoidance of interspecific matings, a low fitness of hybrids, or both. These results suggesting the presence of very effective barriers to gene flow between the two species are robust to uncertainties about the exact demography of the Paleolithic populations, and they are also found to be compatible with the observed lack of mtDNA introgression. Our model additionally suggests that similarly low levels of introgression in Europe and Asia may result from distinct admixture events having occurred beyond the Middle East, after the split of Europeans and Asians. This hypothesis could be tested because it predicts that different components of Neanderthal ancestry should be present in Europeans and in Asians.