Stationary Models of Population Structure Fail to Explain Observed Declines in Neanderthal and DenisovanNe

Abstract

AbstractIn a subdivided population, effective population size (Ne) reflects not only the number of individuals, but also the rate and pattern of gene flow. If we sample two haploid genomes from the same deme, estimates ofNewill be small in the recent past but large in the distant past, even if there were no changes in census size or in the rate or pattern of gene flow. This article asks whether this effect can plausibly explain observed declines in effective sizes of archaic populations.Archaic human populations (Neanderthals and Denisovans) had very low heterozygosity. If this value represents a stationary equilibrium and migration is conservative, then heterozygosity alone allows us to estimate that each of these populations had a total effective size (including all subdivisions) of about 3600 individuals.Both archaic populations also exhibit a pronounced decline inNeover a period of about 400 thousand years. Under the finite island model of population structure, these data, together with a total population size of 3600, imply an implausible level of separation among archaic demes. Convergence is slower under models of isolation by distance. Thus, we might expect such models to provide a better fit to the slow observed decline. Yet even the most extreme form of isolation by distance, which restricts gene flow to a single dimension, implies an implausible level of separation among demes. Furthermore, these models all imply that the ratio of early to lateNeis much greater than that observed in archaic humans.It seems unlikely that any model assuming stationary equilibrium can explain the pattern observed in archaic DNA. That pattern apparently reflects change either in the number of individuals or in the rate or pattern of gene flow.