The role of deep hybridization in fern speciation: Examples from the Thelypteridaceae

Hybridization is recognized as an important mechanism in fern speciation, with many allopolyploids known among congeners, as well as evidence of ancient genome duplications. Several contemporary instances of deep (intergeneric) hybridization have been noted, invariably resulting in sterile progeny. We chose the christelloid lineage of the family Thelypteridaceae, recognized for its high frequency of both intra- and intergeneric hybrids, to investigate recent hybrid speciation between deeply diverged lineages. We also seek to understand the ecological and evolutionary outcomes of resulting lineages across the landscape.

By phasing captured reads within a phylogenomic data set of GoFlag 408 nuclear loci using HybPhaser, we investigated candidate hybrids to identify parental lineages. We estimated divergence ages by inferring a dated phylogeny using fossil calibrations with treePL. We investigated ecological niche conservatism between one confirmed intergeneric allotetraploid and its diploid progenitors using the centroid, overlap, unfilling, and expansion (COUE) framework.

We provide evidence for at least six instances of intergeneric hybrid speciation within the christelloid clade and estimate up to 45 million years of divergence between progenitors. The niche quantification analysis showed moderate niche overlap between an allopolyploid species and its progenitors, with significant divergence from the niche of one progenitor and conservatism to the other.

The examples provided here highlight the overlooked role that allopolyploidization following intergeneric hybridization may play in fern diversification and range and niche expansions. Applying this approach to other fern taxa may reveal a similar pattern of deep hybridization resulting in highly successful novel lineages.