Five-leaf Generalizations of the D-statistic Reveal the Directionality of Admixture

Area of Science:

• Population Genetics
• Phylogenetics
• Genomics

Background:

• The D-statistic is a widely used four-taxon test for detecting organismal admixture (hybridization or introgression) using single nucleotide polymorphism data.
• While the D-statistic can identify admixture, it struggles to determine the directionality of gene flow using only four-taxon models.
• Existing five-taxon methods like DFOIL have limitations, particularly with ancient samples and full utilization of tree symmetry.

Purpose of the Study:

• To develop a new, general family of five-taxon admixture tests, termed Δ-statistics, capable of inferring admixture directionality.
• To introduce new tree topologies ('asymmetric' and 'quasisymmetric') to supplement existing models for admixture analysis.
• To address the limitations of previous methods, including the handling of ancient samples and singleton allelic patterns.

Main Methods:

• Development of a novel family of five-leaf admixture tests (Δ-statistics) inspired by the DFOIL method.
• Incorporation of 'asymmetric' (A) and 'quasisymmetric' (Q) tree models alongside the existing 'symmetric' (S) model.
• Testing the consistency of Δ-statistics using simulated data and empirical data from black, brown, and polar bears, including ancient samples.

Main Results:

• The new Δ-statistics provide a general framework for admixture detection and directionality inference, adaptable to various sampling strategies.
• Analysis of bear data using the symmetric tree (S) revealed bidirectional gene flow between polar and brown bears, contrary to previous claims of unidirectional introgression.
• Application of Δ-statistics on asymmetric (A) and quasisymmetric (Q) trees suggested two distinct admixture phases: ancient brown bear to polar bear gene flow, followed by recent polar bear to brown bear introgression.

Conclusions:

• The developed Δ-statistics offer a robust and versatile tool for studying admixture directionality across diverse evolutionary scenarios, including those with ancient DNA.
• The findings challenge previous interpretations of bear hybridization, highlighting complex bidirectional gene flow and multiple admixture events.
• This work advances the field of population genomics by providing improved methods for resolving complex demographic histories and introgression patterns.