Robust model selection between population growth and multiple merger coalescents.

Area of Science:

• Population Genetics
• Computational Biology
• Evolutionary Modeling

Background:

• Distinguishing between different models of population history is crucial in evolutionary studies.
• Kingman coalescents and Ξ-coalescents represent distinct scenarios of ancestral recombination and merger events.
• Previous methods showed promise but required simplification of population structures.

Purpose of the Study:

• To evaluate the robustness of the singleton-tail statistic in distinguishing between Kingman and Ξ-coalescents under biological confounders.
• To assess the impact of population structure, recombination, and selection on model selection power.
• To explore the utility of the singleton-tail statistic for differentiating multiple merger scenarios.

Main Methods:

• Utilized a computationally tractable summary statistic, the singleton-tail statistic.
• Performed approximate likelihood ratio tests to compare model classes.
• Simulated data under various scenarios including population growth, recombination, selection, and population structure.

Main Results:

• The singleton-tail statistic maintains high power for distinguishing Kingman and Ξ-coalescents when recombination and selection are present.
• Misspecification of population structure significantly reduces the power of the singleton-tail statistic.
• The statistic demonstrated moderate power (up to 30%) in differentiating multiple mergers arising from selective sweeps versus high fecundity.

Conclusions:

• The singleton-tail statistic is a powerful tool for model selection in population genetics, robust to several biological confounders.
• Careful consideration of population structure is essential when applying this statistic.
• The method offers a promising approach for resolving complex demographic histories involving multiple merger events.