A deterministic model of admixture and genetic introgression: the case of Neanderthal and Cro-Magnon

There is an ongoing debate in the field of human evolution about the possible contribution of Neanderthals to the modern human gene pool. To study how the Neanderthal private alleles may have spread over the genes of Homo sapiens, we propose a deterministic model based on recursive equations and ordinary differential equations. If the Neanderthal population was large compared to the Homo sapiens population at the beginning of the contact period, we show that genetic introgression should have been fast and complete meaning that most of the Neanderthal private alleles should be found in the modern human gene pool in case of ancient admixture. In order to test/reject ancient admixture from genome-wide data, we incorporate the model of genetic introgression into a statistical hypothesis-testing framework. We show that the power to reject ancient admixture increases as the ratio, at the time of putative admixture, of the population size of Homo sapiens over that of Neanderthal decreases. We find that the power to reject ancient admixture might be particularly low if the population size of Homo sapiens was comparable to the Neanderthal population size.