Diagnosis of the mechanisms of different types of discordances between phylogenies inferred from nuclear and mitochondrial markers

Abstract

In ancient freshwater lakes, an abnormally large species diversity is observed. The mechanisms that g nerated extremely high biodiversity in the ancient lakes have not been sufficiently studied and remain only partially known. Sequences of environmental changes in highly complex ecosystems such as Lake Baikal, may induce sophisticated combinations of microevolutionary processes. These processes are likely to result in unusual “patterns” of genetic variability of species. The most unusual patterns include the ones when speciation is followed by incomplete lineage sorting as well as mitochondrial or nuclear introgression. All these phenomena are diagnosed by comparing the topologies of phylogenetic trees inferred from molecular markers of evolution located in mitochondria and nuclei. Mitochondrial and nuclear introgression is a particularly interesting and complex case, which is the process of incorporating the gene alleles of one species into the gene pool of a sister species due to interspecific hybridization (introgressive hybridization). In many cases, existing methods for molecular phylogenetic analysis do not automatically allow the observed patterns of polymorphism to be explained and, therefore, cannot provide hypotheses that would explain the mechanisms which resulted to these patterns. Here we use adaptive dynamics models to study neutral molecular evolution under various scenarios of interaction between sister species and the environment. We propose and justify a set of criteria for detecting how two evolutionary trees may differ, with a special focus on comparing a tree inferred from nuclear DNA to one from mitochondrial DNA. The criteria react to branching pattern and branch lengths, including relative distances from ancestral lineages. Simulations show that the criteria allow fast and automated detection of various types of introgression, secondary breaches of reproductive barriers, and incomplete lineage sorting.