Assortative mating enhances postzygotic barriers to gene flow via ancestry bundling

Abstract

Hybridization and subsequent genetic introgression are now known to be common features of the histories of many species, including our own. Following hybridization, selection often purges introgressed DNA genome-wide. While assortative mating can limit hybridization in the first place, it is also known to play an important role in postzygotic selection against hybrids and, thus, the purging of introgressed DNA. However, this role is usually thought of as a direct one: a tendency for mates to be conspecific reduces the sexual fitness of hybrids, reducing the transmission of introgressed ancestry. Here, we explore a second, indirect role of assortative mating as a postzygotic barrier to gene flow. Under assortative mating, parents covary in their ancestry, causing ancestry to be “bundled” in their offspring and later generations. This bundling effect increases ancestry variance in the population, enhancing the efficiency with which postzygotic selection purges introgressed DNA. Using whole-genome simulations, we show that the bundling effect can comprise a substantial portion of mate choice’s overall effect as a postzygotic barrier to gene flow. We then derive a simple method for estimating the impact of the bundling effect from standard metrics of assortative mating. Applying this method to data from a diverse set of hybrid zones, we find that the bundling effect increases the purging of introgressed DNA by between 1.2-fold (in a baboon system with weak assortative mating) and 14-fold (in a swordtail system with strong assortative mating). Thus, assortative mating’s bundling effect contributes substantially to the genetic isolation of species.