Coalescent process with fluctuating population size and its effective size.

Area of Science:

• Population genetics
• Stochastic processes
• Mathematical biology

Background:

• The Wright-Fisher model is a cornerstone of population genetics theory.
• Understanding population size fluctuations is crucial for accurate evolutionary modeling.
• Coalescent theory describes the ancestry of genes within a population.

Purpose of the Study:

• To analyze the coalescent process in a Wright-Fisher model with a finite Markov chain for population size.
• To establish the relationship between coalescence time expectation and population size means.
• To define and characterize the coalescence effective population size (cEPS).

Main Methods:

• Development of a sequence of two-dimensional discrete time Markov chains.
• Analysis of the limiting process of these Markov chains.
• Calculation of the Laplace transform for coalescence time distribution.

Main Results:

• Demonstrated that cEPS is strictly larger than the harmonic mean and smaller than the arithmetic mean.
• Showed that cEPS approaches the harmonic mean with faster population size fluctuations and the arithmetic mean with slower fluctuations.
• Derived an explicit expression for cEPS in the case of a two-valued Markov chain, detailing its dependence on sample size.

Conclusions:

• The coalescence effective population size (cEPS) provides a more nuanced measure than simple population means.
• The dynamics of population size fluctuations significantly impact genetic coalescence.
• The findings offer a refined framework for studying genetic drift and diversity in variable populations.