Site pattern probabilities under the multispecies coalescent and a relaxed molecular clock: Theory and applications

The first step in statistical inference of the evolutionary histories of species is developing a probability model that describes the mutation process as accurately and realistically as possible. A major complication of this inference is that different loci on the genome can have histories that diverge from the common species history and each other. The multispecies coalescent process is commonly used to model one source of this divergence, incomplete lineage sorting, or ILS. In Chifman and Kubatko (2015), the authors computed the site pattern probabilities for four taxa under a full probability model based on the Jukes-Cantor substitution model when the molecular clock holds. This paper generalizes that work to a relaxed clock model, allowing for mutation rates to differ among species. This will enable better phylogentic inference in cases where the molecular clock does not hold.