Root probabilities for intraspecific gene trees under neutral coalescent theory

Area of Science:

• Population genetics
• Molecular evolution
• Phylogenetics

Background:

• Rooting intraspecific gene or haplotype trees is difficult with traditional methods like outgroups.
• Existing techniques often face limitations in accurately determining evolutionary relationships within a species.

Purpose of the Study:

• To apply neutral coalescent theory for rooting intraspecific gene trees.
• To determine the relative ages of haplotypes within a population.
• To develop a more effective method for rooting gene trees when traditional outgroups are unsuitable.

Main Methods:

• Utilized neutral coalescent theory and a recursive equation to calculate exact root probabilities.
• Developed and tested a heuristic for determining outgroup weights by identifying the oldest haplotype.
• Employed computer simulations to validate the heuristic and compare it with haplotype frequency.

Main Results:

• Exact root probabilities are not highly sensitive to the parameter theta (related to mutation rate and effective population size).
• Root probabilities depend more on relative haplotype proportions than absolute counts.
• The heuristic of using the oldest haplotype as an outgroup showed strong correlation with actual age, outperforming haplotype frequency.

Conclusions:

• Neutral coalescent theory provides a viable alternative for rooting intraspecific gene trees.
• The oldest haplotype in a sample can serve as a reliable outgroup for phylogenetic analysis.
• This heuristic offers a more accurate method for estimating relative haplotype ages compared to frequency-based approaches.