The Ecology of Hybrid Incompatibilities

Ecologically mediated selection against hybrids, caused by hybrid phenotypes fitting poorly into available niches, is typically viewed as distinct from selection caused by epistatic Dobzhansky-Muller hybrid incompatibilities. Here, we show how selection against transgressive phenotypes in hybrids manifests as incompatibility. After outlining our logic, we summarize current approaches for studying ecology-based selection on hybrids. We then quantitatively review QTL-mapping studies and find traits differing between parent taxa are typically polygenic. Next, we describe how verbal models of selection on hybrids translate to phenotypic and genetic fitness landscapes, highlighting emerging approaches for detecting polygenic incompatibilities. Finally, in a synthesis of published data, we report that trait transgression-and thus possibly extrinsic hybrid incompatibility in hybrids-escalates with the phenotypic divergence between parents. We discuss conceptual implications and conclude that studying the ecological basis of hybrid incompatibility will facilitate new discoveries about mechanisms of speciation.